* VE.Direct: return non-nullptr as a fallback
the changed return statement was supposed to return a shared_ptr to a
new and valid MPPT data struct as a fallback. however, it did return a
new shared_ptr that was initialized to nullptr.
* VE.Direct: make liveview total use total MPPT values
this change makes the call to VictronMppt.getData() obsolete, which in
turn will therefore not cause an error message on the console if
VE.Direct (MPPT) is not enabled. this change also takes care that once
multiple VE.Direct MPPT charge controllers are supported, the sums of
the respective total values are used in the web app totals.
* introduce VictronMpptClass
this solves a design issue where the loop() method of a static instance
of VeDirectMpptController, which is part of library code, is called as
part of the main loop() implementation. that is a problem because the
call to this loop() must be handled differently from all other calls:
the lib does not know whether or not the feature is enabled at all.
also, the instance would not be initialized when enabling the feature
during normal operation. that would even lead to a nullptr exception
since the pointer to the serial implementation is still uninitialized.
this new intermediate class is implemented with the support for multiple
Victron charge controllers in mind. adding support for more charge
controllers should be more viable than ever.
fixes#481.
related to #397#129.
* VE.Direct: move get.*AsString methods to respective structs
those structs, which hold the data to be translated into strings, know
best how to translate them. this change also simplifies access to those
translation, as no parameter must be handed to the respective methods:
they now act upon the data of the instance they are called for. adds
constness to those methods.
* VE.Direct: simplify and clean up get.*AsString methods
use a map, which is much easier to maintain and which reads much easier.
move the strings to flash memory to save RAM.
* DPL: use VictronMpptClass::getPowerOutputWatts method
remove redundant calculation of output power from DPL. consider
separation of concern: VictronMpptClass will provide the total solar
output power. the DPL shall not concern itself about how that value is
calculated and it certainly should be unaware about how many MPPT charge
controllers there actually are.
* VE.Direct: avoid shadowing struct member "P"
P was part of the base struct for both MPPT and SmartShunt controller.
however, P was also part of the SmartShunt controller data struct,
shadowing the member in the base struct.
since P has slightly different meaning in MPPT versus SmartShunt, and
since P is calculated for MPPT controllers but read from SmartShunts, P
now lives in both derived structs, but not in the base struct.
* VE.Direct: isDataValid(): avoid copying data structs
pass a const reference to the base class implementation of isDataValid()
rather than a copy of the whole struct.
* VE.Direct: unify logging of text events
* VE.Direct: stop processing text event if handled by base
in case the base class processed a text event, do not try to match it
against values that are only valid in the derived class -- none will
match.
* VE.Direct MPPT: manage data in a shared_ptr
instead of handing out a reference to a struct which is part of a class
instance that may disappear, e.g., on a config change, we now manage the
lifetime of said data structure using a shared_ptr and hand out copies
of that shared_ptr. this makes sure that users have a valid copy of the
data as long as they hold the shared_ptr.
* VE.Direct MPPT: implement getDataAgeMillis()
this works even if millis() wraps around.
* VE.Direct: process frame end event only for valid frames
save a parameters, save a level of indention, save a function call for
invalid frames.
avoid staleness in case the same power limit is calculated over and over
again, hence no new power limit value is calculated and hence no power
limit command is sent to the inverter. staleness occurs in this case if
the first power limit command to establish the respective limit was not
received by the inverter. one can easily simulate a situation where the
same power limit is caluclated over and over again: with a battery above
the start threshold, set a very low upper power limit for the inverter
(DPL setting). that value will be used as the limit as long as the power
meter reading is larger than that.
we could also check the limit reported by the inverter. however, that
value is in percent of the inverter's max AC output, and is often not
the same value as we requested as the limit, but slightly off. we then
would have to decide how much deviation is okay, which is unreasonably
complicated.
closes#478.
Sunrise and -set must recomputed if one of the following conditions is met:
* The date changed (based on the selected timezone)
* Location (Lat/Lon) changed
* Sunset type changed
So instead of calculating that every minute just do it on update via web interface or date change.
If a new config is uploaded, the DTU gets restarted. There is no need to initiate a recalculation in this case.
* Move Mppt logic to subclass
* Added Definitions for Shunts and restructering
* First integration of SmartShunt data into Web Interface
* Code cleanup
* VE.Direct: whitespace cleanup
* VE.Direct: manage HardwareSerial in unique_ptr
* VE.Direct: _efficiency is only needed by MPPT
* VE.Direct: keep as many members private as possible
* VE.Direct: use int8_t for pins (as before)
* VictronSmartShunt: _verboseLogging is not used
* VE.Direct: OR (off reason) is MPPT specific
it also applies to Phoenix inverters and Smart BuckBoost, but since
there is no support for those, the code is moved to the MPPT controller.
* Added Shunt alarms to liveview
Changed from double to int for several readings
* Update build.yml to allow manual builds
---------
Co-authored-by: Philipp Sandhaus <philipp.sandhaus@cewe.de>
Co-authored-by: Bernhard Kirchen <schlimmchen@posteo.net>
* JK BMS: avoid trailing whitespace in debug output
* JK BMS: publish data points through MQTT
* JK BMS: updateFrom: skip data points with equal value
this changes the interpretation of the timestamp in data containers that
are merely updated from other data containers: this is the oldest
timestamp known where the value was as recorded by the data point in its
respective container.
the data container constructed from an answer will -- naturally -- have
the timetamps of its data points set to the time they were constructed.
* JK BMS: only publish changed values to MQTT broker
all values are still published once every minute if the MQTT retain flag
is NOT set. otherwise, the constant values are only published once on
startup.
* JK BMS: avoid trailing whitespace in debug output
* JK BMS: publish data points through MQTT
* JK BMS: updateFrom: skip data points with equal value
this changes the interpretation of the timestamp in data containers that
are merely updated from other data containers: this is the oldest
timestamp known where the value was as recorded by the data point in its
respective container.
the data container constructed from an answer will -- naturally -- have
the timetamps of its data points set to the time they were constructed.
* JK BMS: only publish changed values to MQTT broker
all values are still published once every minute if the MQTT retain flag
is NOT set. otherwise, the constant values are only published once on
startup.
* thread-safety and dynamic memory for MessageOutput
* use dynamic memory to allow handling of arbitrary message lenghts.
* keep a message buffer for every task so no task ever mangles the
message of another task.
* every complete line is written to the serial console and moved to
a line buffer for sending them through the websocket.
* the websocket is always fed complete lines.
* make sure to feed only as many lines as possible to the websocket
handler, so that no lines are dropped.
* lock all MessageOutput state against concurrent access.
* MessageOutput: respect HardwareSerial buffer size
the MessageOutput class buffers whole lines of output printed by any
task in order to avoid mangling of text. that means we hand over full
lines to the HardwareSerial instance, which might be too much in one
call to write(buffer, size). we now check the return value of
write(buffer, size) and call the function again with the part of the
message that could not yet be written by HardwareSerial.
* VE.Direct: reset state machine on timeout
there will never be a large gap between two bytes of the same frame.
if such a large gap is observed, reset the state machine so it tries
to decode a new frame once more data arrives.
this is helpful in case of corrupted data that prevents the state
machine of transitioning to the final state even though the VE.Direct
data producer is done sending bytes that belong to the same frame.
* VE.Direct: print problems to MessageOutput
this includes the web console in particular, where many users have
access to while the serial console is not attached or monitored.
* VE.Direct: collect serial input into buffer and print
should help debug issues for users.
* VE.Direct: implement and use verbose logging switch
* add Icons for Battery and Victron device sensors in Home Assistant
overriding the boring default icon for many sensors
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* DPL MQTT handler: modernize
* there is no need to tokenize and check the topic of a received MQTT
message if we only subscribe to a single topic. all messages will be
for that topic. avoid testing the topic in the callback alltogether.
* use std::string and std::stoi over allocating and deleting a buffer
and copying charactes around.
* use a switch statement to process the actual payload.
* break a long line.
* DPL: fix getMode() return value
getMode() returned a bool. probably its return type was not adjusted
when the third mode was introduced. this lead to mode 2 being cast to
true implicitly, which in turn was used to construct a String, such that
"1" was published as the DPL mode when in fact it was 2.
make the mode an enum class to avoid such problems in the future.
inline getMode() and setMode().
fix indention.
the size allocated for the HTTP request response was too little, while
the size for the buffer of the websocket output was increased already.
add a new member variable and use it in both context, such that
increasing the buffer size to accomodate more space (for the JSON data
in particular) will benefit both contexts in the future.
* DPL: improve verbose logging
* shorten DPL log prefix
* canUseDirectSolarPower() was printed two times
* _batteryDischargeEnabled was printed two times
* convert boolean values to human-readable strings
* add units where possible
* split messages into block "before calculating new limit" and "after
calculating new limit", as the latter cannot rely on _inverter being
available.
* order messages such that variables whose value is derived from other
variables are printed later than their dependencies.
* merge output into blocks (one instance near "Printout some stats")
* remove more redundant info (produced in functions outside loop())
* print target grid consumption
* DPL: inhibit solar passthrough while stop threshold reached
* DPL: implement and use isBelowStopThreshold()
we only want to inhibit solar passthrough if the SoC is *below* the stop
threshold, not if it is equal to the stop threshold. otherwise, when
discharging, we would discharge until the battery reached the stop
threshold, then we would also inhibit solar passthrough, until the
battery is charged to the SoC stop threshold plus one percent.
processing a published valued on a subscribed topic is currently running
in a task that is not the task executing the main loop(). that's because
the espMqttClient(Secure) was constructed without arguments, which
selects the constructor with two arguments priority and core, both of
which have default values. that constructor selects
espMqttClientTypes::UseInternalTask::YES, causing a task to be created
in which context the MQTT client loop is executed.
MQTT subscribers assume they are running in the same context as the main
loop(). most code assumes exactly that. as the scheduler is preemptive
and very little (none at all?) code is interlocked, we have to make sure
to meet the programmer's expectations.
this changeset calls the MQTT client loop in the context of the main
loop() and enforces the use of espMqttClientTypes::UseInternalTask::NO.
* PowerMeter: gracefully handle non-float MQTT values
* PowerMeter: update _lastPowerMeterUpdate conservatively
update the timestampt only if the topic actually matched any
subscription and if the value could be parsed as a float.
* PowerMeter: unsubscribe before subscribing
* PowerMeter: organize subscriptions in a map
this allows for a slightly more elegant code and reduced amount of code
overall.
* PowerMeter: clean up header
* move private methods to private section of class declaration.
* remove unused member variable.
Make the administrative accesspoint timeout configurable. The default
value is 3 minutes, values from 0-99999 are possible, where 0 means
infinite (no timeout).
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* DPL: implement verbose logging switch
* MQTT: implement verbose logging switch
* power meter: implement verbose logging switch
* Hoymiles lib: implement verbose logging switch
* cpp linting: "final" makes "virtual" and "override" redundant
... however, using only "final" is not as verbose.
When OpenDTU has a Pylontech CAN Bus Battery connected and enabled, this
patch adds the discovery routine for Home Assistant
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
if the new calculated power limit is below the minimum power limit
setting, the inverter is shut down. the shutdown() function is called
every time this condition is detected, which is also true if the
inverter is kept shut down for longer. that happens while the battery
is charging in particular (solar passthrough off). there are other
cases.
in such cases we still want to get into the DPL status "stable". to be
able to determine this stable state, we must know if the call to
shutdown did actually initiate a shutdown or if the inverter is already
shut down.
we then can forward this "changed" or "not changed" info up the call
chain, where the loop() will know that the system is actually stable.
* fix another fixable "passtrough" typo
the typo in the config's identifier is not changed to preserve
compatibility while not spending the effort to migrate the setting.
* webapp language: prefer SoC over SOC
* DPL: implement solar passthrough loss factor
in (full) solar passthrough mode, the inverter output power is coupled
to the charge controler output power. the inverter efficiency is already
accounted for. however, the battery might still be slowly discharged for
two reasons: (1) line losses are not accounted for and (2) the inverter
outputs a little bit more than permitted by the power limit.
this is undesirable since the battery is significantly drained if solar
passthrough is active for a longer period of time. also, when using full
solar passthrough and a battery communication interface, the SoC will
slowly degrade to a value below the threshold value for full solar
passthrough. this makes the system switch from charging the battery
(potentially rapidly) to discharging the battery slowly. this switch
might happen in rather fast succession. that's effectively
trickle-charging the battery.
instead, this new factor helps to account for line losses between the
solar charge controller and the inverter, such that the battery is
actually not involved in solar passthrough. the value can be increased
until it is observed that the battery is not discharging when solar
passthrough is active.
In Home Assistant, when Home Assistant MQTT-Auto-Discovery is active,
almost all Sensors of the auto-discovered Victron device in Home
Assistant become "unavailable" after a short time - except those
Sensors with frequent changes like battery voltage or panel voltage.
This patch introduces regular mqtt updates for all VE.Direct sensors
when MQTT-Auto-Discovery is enabled.
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* DPL: wait for valid time information
we know that the Hoymiles library refuses to send any message to any
inverter until the system has valid time information. until then we can
do nothing, not even shutdown the inverter.
* DPL: wait for device info to be ready
a calculated power limit will always be limited to the reported
device's max power. that upper limit is only known after the first
DevInfoSimpleCommand succeeded. wait for that information to be
available.
* DPL: fix initial calculcation backoff
if the calculation backoff is initialized to zero, the backoff will be
doubled to zero until a new, different power limit was calculated for
the first time. this lead to the DPL recalculating a power limit
hundreds of times without a backoff after startup.
* VE.Direct: remove polling interval
the polling interval was meant to limit the amount of MQTT updates.
however, that is already controlled by the global MQTT publish interval.
the removed interval was instead used to limit polling of the VE.Direct
UART for incoming data.
the Victron device sends data unsolicited. the VeDirectFrameHandler does
not implement any polling mechanism. no data is ever sent to the Victron
device.
what the removed polling interval did was cause a buffer overrun of the
HardwareSerial class, since the incoming data was not processed in time.
so every five seconds, we read a whole valid VE.Direct frame, plus some
old data, which was not a whole frame, leading to VE.Direct error
messages to pop up.
with the polling interval removed, no framing errors are reported, and
instead we gain new data from the charge controller approximately ever
two seconds -- for free.
* VE.Direct: change texts to correct VE.Direct capital letters
* VE.Direct: improve "UpdatesOnly" switch labels
especially since the publish interval setting is gone, the label makes
it hard to comprehend what the switch does. update the texts to better
explain what the switch is used for.
use the same text on the VE.Direct info view.
* VE.Direct: use StatusBadge on info view
there were custom badges to indicate the VE.Direct settings. replace
those by the common StatusBadge to make then look the same as the other
badged on the info views.
a new status is needed to communicate that no update was sent to the
inverter because its power limit is still valid. in this case,
calculating a new power limit is delayed by an exponentially increasing
backoff. the maximum backoff time is ~1s, which is still plenty fast.
the backoff is actually necessary for another reason: at least
currently, a lot of debug messages are printed to the console. printing
all that information in every DPL loop() is too much.
the unconditional solar passthrough mode, configured using MQTT, works
differently than the normal mode of operation. it is also independent
from the power meter reading. if this mode is active, a shortcut is
taken to a function that implements the actions for this mode. this is
convenient since we don't have to consider special cases in the code
that handles normal mode of operation.
the DPL already took care to shut down the inverter if anything fishy
was going on, mainly to make sure that the battery is not drained.
however, some cases were missed:
* if the configuration changed such that another inverter is now
targeted, the one the DPL controlled previously was not shut down.
* if the configuration changed such that another inverter (different
serial number) was configured at the same index, the previous one
was not shut down.
this change corrects these problems by making the DPL keep a copy of the
shared_ptr to the inverter. the shared_ptr is only released once the DPL
shut the respective inverter down.
this implementation checks all requirements for a new power limit to be
calculated, one after the other. if any requirement is not met, a
respective status is announced.
status messages are communicated on the (serial) console. these can also
be displayed easily on the web app in the future. the status texts
explain clearly what the DPL is currently doing, which aids
understanding how the DPL works. the status is only announced if it
changes, or after a fixed interval.
as each requirement is checked individually, the code readability is
improved as well. previously, all the respective conditions had to be
checked as well, but the statements were more complex.
the DPL loop is now executed with high frequency, i.e., it does not wait
for a fixed timespan to pass before checking requirements. it always
aborts on the first unmet requirement. this should improve responsiveness,
as the DPL checks all requirements more often.
the DPL now waits for all power commands and power limit updates to
complete. when that is the case, a settling time elapses. after the
settling phase, the DPL waits for a new update from the inverter and
from the power meter. now it can be assumed that the values are in sync.
it then makes sense to calculate a new power limit immediately, which
the DPL then does.
the defaults for solar passthrough voltage thresholds shall be floats,
so the user can store float values to the config. otherwise, float
values can and will be stored, but when reading them, the defaults will
be applied as the defaults are of an incompatible type.
there is no need to assume and hardcode a fixed efficiency for the
Victron solar charger. the charger reports the voltage and current at
its battery terminal, which can be used to calculate the charger's
actual power output.
the fallback to 100% for the efficiency of the Hoymiles inverter, in
case it is not producing power, is too optimistic. this commit proposes
to use 96.7% as the efficiency for that case, which is the peak
efficiency for many (all?) Hoymiles inverters as per datasheet. that
value should be closer to the real efficiency that will be achieved once
the inverter is turned on.
In addition to the cyclic query of the power meters, they will be queried on demand when total power is requested and last update is older than 1 second.
Webinterface change to set full solar passthrough values
Adding webapi and config changes to enable full solar passthrough over certain battery Soc
inital version of full solar passthrough in power limiter
Passthrough mode can be enabled via MQTT
translations
re-enable comment
remove unused variable
rename function WebApiMqttClass::getRootCaCertInfo to more generic
name WebApiMqttClass::getTlsCertInfo
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
User asked for TLS client certificate based login from DTU to MQTT
server. This PR implements storage and use of x509 client certificate
and private key.
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* add support for energy & power readings on SML based power meters, taking OBIS 16.7.1 for power (using mod. SML Parser lib. by olliiiver)
* switched SML read to use software serial
* made total power meter response controled by meter source to obtain either the sum of phase powers or explicit total power provided by meter
* made mqtt subscriptions to power meter topics meter source dependend
* simplified SML read loop and OBIS handler registration, + minor refactoring
* minor cleanup/style changes and optim. PowerMeter
* fixed build, add SOURCE_SML == 4
* removed optional usage of HW serial for SML power meter
* switched to usage of _powerMeter1Power for SML power reading to allign better with existing code
---------
Co-authored-by: helgeerbe <helge@erbehome.de>
* Power limiter: Use the actual AC power for limit calculation
instead of the last set limit.
In order support setups without battery connected (sources that don't exhaust the limit)
* adding data age to battery data
* Add battery enabled flag
* Webapi and websocket api for Battery
* Webinterface for battery
* fixed bug due to naming inconsistencies
* cleaned up rounding
* dist update
* change typename to uppercase
* reverting to original file
This allows to enter a offset in kWh in the inverter properties which will be applied to the read Yield Total value of the inverter. Using this can set your total production to zero if you e.g. are using a used device.
Added additional field to the prometheus api which identifies a channel by it's type. That means that e.g. channel 0 exists for type AC and DC.
This commit also introduces a additional field in the statistics byte assignment table. This field identifies whether a channel is on the AC or DC side. MQTT and WebAPI is still compatible with the previous design.
chango to full Buffer Mode, remove picture loop,
because it's not worked.
look: https://github.com/olikraus/u8glib/wiki/tpictureloop
snprintf + public vars
- dispPowerSafe => true: PowerSafe Mode on
- dispLogo => true: showing Logo upper right corner
- dispContrast => possible to adjust the Display Contrast
change in date time, buffer deleted
Display-Constructor changed, reset goes to front
Arrow and Logo only shown, when production is on
Refactor OLEDDisplayClass to DisplayGraphicClass
After this commit its not possible to migrate from the old binary blob config to the new json based config!! If you still running a old version before 12. October please upgrade to a version before this commit.
See https://github.com/tbnobody/OpenDTU/discussions/285
