* remove/comment unused variables to avoid compiler warnings
* cleanups: fix indention and style, make variable private, implement
getters in header and make const.
* optimize message output: respect verbose logging setting. prefix
output with SMA_HM.
* use newly introduced mutex in PowerMeterClass also for SMA HomeManager
* refactor code for readibility, unindent where possible.
the SDM power meter (among others) writes the power consumption of three
phases in multiple steps. this change helps to prevent getTotalPower()
reading intermediate values, e.g., reading a new value for phase 1 but
old values for phase 2 and 3 since phase 2 is currently read.
cache the values, and write them all at once, protected by a mutex,
later.
closes#732.
this changeset refactors the web application's DPL settings view. the
DPL settings can be complex, and they shall be presented in a way that
allows users to comprehend their meaning. irrelevant settings are now
hidden or displayed dynamically based on the influencing settings.
* group SoC thresholds into their own card
* hide battery SoC thresholds if battery disabled. if the user did not
even enable the battery interface, battery SoC values will not be used
for DPL decisions. in that case we completely hide the respective
settings from the DPL admin view. this reduces the amount of settings
for new users and especially users who don't even have a battery in
their setup or have no BMS connected.
* group voltage thresholds and improve label texts
* fix load correction factor unit
* fix header (wording)
* group solar-passthrough settings in new card
* group inverter-related settings
* hide solar passthrough settings if VE.Direct is disabled. closes#662.
* completely disable form if any requirement is not met
* list available inverters by name and type. this makes it much more
convenient to select the right inverter, especially since the order of
the inverters in the web UI is decoupled from their position in the
internal array, which was used to select them previously. care was
taken that old configs select the same inverter after an update.
when editing the DPL settings, the selects an inverter from the newly
created drow-down list, and the respective old inverter is
pre-selected.
* disable form if no inverter is configured (config alert)
* make inverter input selection dynamic. adjust selection to actual
amount of channels for selected inverter. skip selection altogether if
inverter has only one channel, or if it is solar powered.
* web app: wording adjustments
* group meta data into new property and exclude from submission. saves
memory when evaluating the submitted settings.
* hide irrelevant settings if inverter is solar-powered
* move restart hour setting to inverter card. translate setting which
disabled automatic restart.
* simplify "drain strategy" setting into an on/off toggle. care was
taken that existing configs work the same after an upgrade. the
respective drain strategy is translated into the new setting when
reading the config. once the config is written, the new setting is
persisted and the old is not part of the config any more.
* show more configuration hints, depending on actual configuration
* replace inputs by InputElement components where possible
* fix compiler warning in SerialPortManager.cpp: function must not
return void
* clean up and simplify implementation of usesHwPort2()
* make const
* overrides are final
* default implementation returns false
* implement in header, as the implementation is very simple
* rename PortManager to SerialPortManager. as "PortManager" is too
generic, the static instance of the serial port manager is renamed to
"SerialPortManager". the class is therefore renamed to
SerialPortManagerClass, which is in line with other (static) classes
withing OpenDTU(-OnBattery).
* implement separate data ages for MPPT charge controllers
* make sure MPPT data and live data time out
* do not use invalid data of MPPT controlers for calculations
* add :key binding to v-for iterating over MPPT instances
this change adds support for a second Victron MPPT charge controller
using a second serial connection.
* Add device configuration for a second victron mppt
* Update VedirectView for second victron mppt
* Update MqttHandleVedirect for second victron mppt
* Update MqttHandleVedirectHass for second victron mppt
* Handle nonexisting victron controllers with optionals
* Add bool-function to Battery and inherited classes, if uart port 2 is
being used
* Introduced a serial port manager. In order to prevent the battery and
the Victron MPPT to use the same hw serial ports, this class keeps
track of the used ports and their owners.
now that users can tell the DPL that their inverter is not powered by a
battery but powered by solar panels, we shall not restart inverters to
reset the daily yield value, if they are solar powered. these inverters
will reboot every night by themselves.
* fix logic in HomeAssistent handler
* also publish voltage thresholds (not just SoC thresholds)
* do not publish irrelevant thresholds to MQTT. if the inverter is
solar-powered, no thresholds are effectively in use by the DPL and it
therefore makes no sense to publish them to the broker. similarly, if
no battery interface is enabled or the SoC values are set to be
ignored, the SoC thresholds are effectively not in use and will not be
published to the broker.
* make HA auto-discovery expire. this makes auto-dicovered items
disappear from Home Assistent if their value is no longer updated.
changes to settings which cause other thresholds to be relevant will
then be reflected in Home Assistent even if some thresholds are no
longer maintaned in MQTT.
* force HA update when related settings change enabling VE.Direct shall
trigger an update since solar passthrough thresholds become relevant.
similarly, enabling the battery interface makes SoC thresholds become
relevant. there are more settings in the power limiter that also
influence the auto-discoverable items.
* break very long lines
the allocated memory to create the JSON with onBattery-specific totals
for the live view was too little to contain all values, which are sent
regularly.
avoid shutting down the inverter at all if the calculated power limit
falls below the lower power limit or if the power meter value is
outdated. do this only if the inverter is setup to be solar powered.
by default and until this change, we assumed that the inverter
controlled by the DPL is powered by a battery. not all users have a
battery in their system. they still use the DPL to achieve net-zero
export. those users can now tell the DPL that their inverter is powered
by solar modules rather than a battery and the DPL will behave
accordingly.
do not scale limit if inverter is not producing, as DC channel power is
expected to be close to zero anyways.
do not scale limit if current inverter limit is small, such that
channels might produce very little power exactly because the limit is so
low.
move the calculation out of setNewPowerLimit and into a new function, so
that we can make use of return statements there.
the update frequency of Victron MPPT charger data, the battery Soc, the
huawei charger power, and the power meter differ from one another, and
differ in particular from the inverter update frequency.
the OnBattery-specific data is now handled in a new method, outside the
upstream code, which merely call the new function(s). the new function
will update the websocket independently from inverter updates. also, it
adds the respective data if it actually changed since it was last
updated through the websocket.
for the webapp to be able to recover in case of errors, all values are
also written to the websocket with a fixed interval of 10 seconds.
we previously only called commitPowerLimit() if the desired limit
changed such that the change was bigger than the hysteresis. we found
that if the limit update was not received and the desired limit would
not change much, the limit of the inverter was wrong for a long time.
to mitigate this, we introduced re-sending the limit update every 60
seconds, regardless of what the limit reported by the inverter was at
that time.
if the power-up command was not received, we also would repeat it only
once every 60 seconds.
this leads to a new kind of staleness and the actual inverter state was
still not matching the desired state.
this new approach effectively adds an additional control loop at the
start of the DPL loop(). that new function compares the requested
inverter state to the actual reported state. it sends updates (limit
update or power on state) until the desired inverter state is reached,
or until a (hard-coded) timeout occurs.
this approach also allows us to send power-up, power-down, and limit
update commands independent from one another and in a particular order.
this should make sure that the inverter is in the desired state even if
conditions change slowly and commands were not received as expected.
* make efficient use of available display area
fix calculation of the text baselines, using getAscent() in favor of
getMaxCharHeight(), which includes ascent and descent. this moves the
first text up and allows to insert margin between the lines until the
display area is fully utilized.
on large displays, if the small diagram is selected, keep the first line
rather low to avoid collision with the diagram y-axis label. in this mode,
there is still more space between the text lines as before, allowing for
improved readability.
* Feature: show power grid usage on display
if the power meter is enabled, the display will use two of three out
of every three-second time slot to show the grid consumption.
closes#620.
the DPL is interested in the battery's voltage to make decisions about
draining the battery or letting it charge (if the user opts to use
voltage thresholds rather than SoC thresholds). using the DC input
voltage reported by the inverter under control has disadvantages:
* the data might be quite old due to the communication protocol
implementation. more inverters being polled means even more lag. the
connection being wireless makes this even worse, due to the need
to retry the occasional lost packet, etc.
* the data is not very accurate, since the DC input of the inverter is
actually some cabling and a couple of junctions away from the actual
battery. this voltage drop can mostly only be estimated and is worse
with higher load. the load correction factor is there to mitigate
this, but it has its own problems and is cumbersome to calibrate.
instead, this change aims to use more accurate battery voltage readings,
if possible. the DPL now prefers the voltage as reported by the BMS,
since it is for sure the closest to the battery of all measuring points
and measures its voltage accurately regardless of the load (the voltage
reading will still drop with higher loads, but this will be only due to
the battery's internal resistance, not that of cabling or junctions). if
no BMS voltage reading is available, the DPL will instead use the charge
controller's voltage reading, as it is available with much higher
frequency and is assumed to be more accurate as it offers a resolution
of 10mV. only if none of these two sources can be used, the inverter DC
input voltage is assumed as the battery voltage.
closes#655.
the Victron SmartShunt communicates the SoC value in permille. this
should be displayed in the web UI accordingly. this is a good excuse to
fully move ownership of the SoC value to the BatteryStats base class and
add a precision indicator variable. this is required to be set each time
a derived class (a battery provider) wants to update the SoC value. the
precision is then used when populating the JSON data for the web UI
(live view).
related to #573.
in the respective context, the DPL only needs to be sure that the SoC
value is not outdated. it should not even care about other values
reported by the battery interface. hence, the isValid() method shall be
concerned with the SoC value timestamp only. the method is renamed for
clarity.
this extends the MqttBattery implementation by an additional topic which
allows to subscribe to receive battery voltage readings through the MQTT
broker. similar to the battery SoC topic, this allows to import a
critical battery data point for the DPL, in case the user chooses to use
voltage thresholds rather than SoC thresholds to control the DPL. if an
otherwise incompatible BMS is available which publishes the battery pack
voltage through MQTT, this can now be used to feed accurate voltage
readings to the DPL.
the BatteryStats base class shall be able to tell the total battery pack
voltage. for that reason, and to avoid code duplication, the voltage is
now handled in the base class and treated as a datum that is common to
all battery providers.
unfortunately, the battery SoC values reported by battery BMSs are
unreliable, at least for some users, or at least without regular
(manual) full charge cycles to calibrate the BMS. it offers great
advantages to connect OpenDTU-OnBattery to a BMS (MQTT publishing of
values, Home Assistent integration, etc.), but previously the users
were then forced to configure the DPL by SoC values.
this change allows to configure the DPL such that SoC values are
ignored. instead, the voltage limits are used to make DPL decisions, as
if no SoC was available in the first place.
the SoC related setting are hidden from the DPL settings view if SoC
values are configured to be ignored.
closes#654.
this implementation avoids the display content jumping the full
screensaver offset from right to left when the modulo operator
wraps. this change makes the display content walk from right to
left as it did walk from left to right.
in case the total production is larger than 1 MWh, i.e., 1000 kWh, the
text on the respective line becomes too large such that it reaches out
of the display when the screensaver is enabled.
this happens on the small and large displays.
this change switches the number format to a float without decimal places
if the total production is larger or equal to 1000 kWh. this saves a dot
and a digit, making the text short enough to fit the display even when
the screensaver moved the display contents as far to the right as it
does.
fix calculation of the text baselines, using getAscent() in favor of
getMaxCharHeight(), which includes ascent and descent. this moves the
first text up and allows to insert margin between the lines until the
display area is fully utilized.
on large displays, if the small diagram is selected, keep the first line
rather low to avoid collision with the diagram y-axis label. in this mode,
there is still more space between the text lines as before, allowing for
improved readability.
* pylontech HA integration: remove unused method/variable
* make MqttHandlePylontechHassClass::publishConfig() private.
there are no outside users of that method.
* rename to MqttHandleBatteryHass
* battery HA integration: merge methods and bring back forceUpdate().
even though the forceUpdate() method was not in use before, it makes
sense to implement it and use it when the battery config changes.
rather than controlling a separate flag, it now changes the _doPublish
flag of the class, which also triggers publishing the device config to
Home Assistant when an MQTT connection problem was detected. since
both situations are now handled similarly, we can merge the loop() and
publishConfig() methods.
* battery: provider specific sensors for HA
* move Battery MQTT loop to BatteryStats
the BatteryStats class should handle the MQTT publishing, including the
interval. for the calculation of a reasonable Home Assistent expiration
value this class now also knows the maximum publish interval.
* JK BMS: fix publishing values for Home Assistent
Home Assistent values expire, because we set them to expire after three
MQTT publish durations. for that reason, we need to re-publish all
values after our self-inflicted full publish interval.
* define JK BMS sensors for Home Assistent
closes#482.
previously, the Pytlontech Home Assistent class implementation had an
init() method, that was never called, as it did nothing. the class
relied on its loop() method being called from the main loop(). after
switching to the TaskScheduler approach, the Pylontech Home Assistent
class init() method was adjusted to register a task that calls the
loop() method periodically. however, the init() method was still not
called.
To reduce the heap usage it is necessary to send the inverters one by one instead of a huge response. A simple call to `/api/livedata/status` returns just some very general information. If detailed inverter information are required the inverter serial number has to appended `?inv=<serial number>`.
The websocket also returns only one inverter at a time. It as to be assembled at client side.
currently the whole SmartShunt data structure is copied to the
BatteryStats instance in every loop, even though the data cannot
possibly have changed. this is quite an expensive task to do in every
loop. this change tracks the last update timestamp and only does the
copy operation if an actual updated data structure was received from
the smart shunt.
this change adds the values of ESP.gteMaxAllocHeap() and
ESP.getMinFreeHead() to the prometheus metrics and the system
information object. the web UI uses these values to diplay the size of
the largest free contiguous block, calculate a rough estimate for the
level of fragmentation, and the maximum usage of heap memory since boot
in absolute and relative amounts.
this change adds the values of ESP.gteMaxAllocHeap() and
ESP.getMinFreeHead() to the prometheus metrics and the system
information object. the web UI uses these values to diplay the size of
the largest free contiguous block, calculate a rough estimate for the
level of fragmentation, and the maximum usage of heap memory since boot
in absolute and relative amounts.
OpenDTU console gets spammed with "WifiGeneric::hostByName() error when first trying to resolve the hostname via DNS. So reverse order: first try mDNS, if that fails try DNS. Also ensure that https bool is passed correctly to HTTPClient::begin(). Lastly, concatenate strings for building Digest authorization using "+" and not via snprintf.
this change utilizes some of the features from library "frozen", which
was included upstream for the grid profile parser. to improve code
maintainability, a couple of std::maps mapping strings to values or the
other way around were introduced in OpenDTU-OnBattery-specific code at
the expense of some flash and computing overhead.
library "frozen" offers constexpr versions of map and string, which
saves initialization code and offers slightly faster lookups. this
brings the binary size down by ~25kB and should provide a small
performance improvement at runtime.
Added ability to deal with local host names (mDNS), remove use of FirebasedJson to save ~20kB build size, some changes to PowerLimiter to avoid setting new inverter power limits when not needed (=current limit as reported by inverter is within hysteresis)
this battery provider implementation subscribes to a user-configurable
MQTT topic to retrieve the battery SoC value. the value is not
re-published under a different topic. there is no card created in the
web app's live view, since the SoC is already part of the totals at the
top of the live view. that is the only info this battery provider
implements.
closes#293.
relates to #581.
MQTT message callbacks are executed in the MQTT thread context. when
processing topics that control the DPL, we must avoid executing methods
that are not thread-safe. this change binds the methods to be called to
the respective parameters and executes them in the TaskScheduler
context, such that they no longer need to be thread-safe.
* bind the callback to a topic (enum value) such that there is no need
to tokenize the full topic (string) to find out what value is being
processed. tokenizing is expensive.
* get rid of using the config in the callback, which improves
thread-safety since the MQTT callback is running in the MQTT thread.
* prefer C++ method stof to convert MQTT value to a float, which saves
us from using new and delete for a buffer in particular.
* prefer switch statements over if-else-trees.
* split long lines.
* get rid of topic #defines.
* fix indention.
MQTT message callbacks are executed in the MQTT thread context. when
processing topics that control the huawei AC charger, we must avoid
executing methods that are not thread-safe. this change bound the
methods to be called to the respective parameters and executes them
in the TaskScheduler context, such that they no longer need to be
thread-safe.
there are more interesting values available to display in the live view.
however, adding them made the list of values very long. this can be
mitigated by using a new column/card, which uses the available screen
space nicely on bigger screens.
this commit re-introduces the changes from #418, which were effectively
reverted with d49481097 (merge commit introducing TaskScheduler).
these adjustments are important to guarantee unmangled log messages and
more importantly, to guarantee that all messages from a particular
component are printed to the web console, which most people use to copy
messages from when reporting issues.
* use dynamic memory to allow handling of arbitrary message lenghts.
* keep a message buffer for every task so no task ever mangles the
message of another task.
* every complete line is written to the serial console and moved to
a line buffer for sending them through the websocket.
* the websocket is always fed complete lines.
* make sure to feed only as many lines as possible to the websocket
handler, so that no lines are dropped.
* lock all MessageOutput state against concurrent access.
* respect HardwareSerial buffer size: the MessageOutput class buffers
whole lines of output printed by any task in order to avoid mangling
of text. that means we hand over full lines to the HardwareSerial
instance, which might be too much in one call to write(buffer, size).
we now check the return value of write(buffer, size) and call the
function again with the part of the message that could not yet be
written by HardwareSerial.
the updateSettings method is called from the web server's context and
therefore accesses _upProvider in a different context than the
TaskScheduler. the lock_guard needs to protect _upProvider.
* Optimize Sun data calculation
* Remove not required enum
* Split config struct into different sub structs
* Feature: Allow configuration of LWT QoS
* Made resetreason methods static
* Feature: Implement offset cache for "YieldDay"
Thanks to @broth-itk for the idea!
Fix: #1258#1397
* Add Esp32-Stick-PoE-A
* remove broken LilyGO_T_ETH_POE config, use device profile instead
* Feature: High resolution Icon and PWA (Progressive Web App) functionality
Fix: #1289
* webapp: Update dependencies
* Initialize TaskScheduler
* Migrate SunPosition to TaskScheduler
* Migrate Datastore to TaskScheduler
* Migrate MqttHandleInverterTotal to TaskSchedule
* Migrate MqttHandleHass to TaskScheduler
* Migrate MqttHandleDtu to TaskScheduler
* Migrate MqttHandleInverter to TaskScheduler
* Migrate LedSingle to TaskScheduler
* Migrate NetworkSettings to TaskScheduler
* Migrate InverterSettings to TaskScheduler
* Migrate MessageOutput to TaskScheduler
* Migrate Display_Graphic to TaskScheduler
* Migrate WebApi to TaskScheduler
* Split InverterSettings into multiple tasks
* Calculate SunPosition only every 5 seconds
* Split LedSingle into multiple tasks
* Upgrade espMqttClient from 1.4.5 to 1.5.0
* Doc: Correct amount of MPP-Tracker
* Added HMT-1600-4T and HMT-1800-4T to DevInfoParser
Fix#1524
* Adjusted inverter names for HMS-1600/1800/2000-4T
* Add channel count to description of detected inverter type (DevInfoParser)
* Adjust device web api endpoint for dynamic led count
* Feature: Added ability to change the brightness of the LEDs
Based on the idea of @moritzlerch with several modifications like pwmTable and structure
* webapp: Update dependencies
* Update olikraus/U8g2 from 2.35.7 to 2.35.8
* Remove not required onWebsocketEvent
* Remove code nesting
* Introduce several const statements
* Remove not required AsyncEventSource
* Doc: Added byte specification to each command
* Feature: Added basic Grid Profile parser which shows the used profile and version
Other values are still outstanding.
* Optimize AlarmLogParser to save memory
* Add libfrozen to project to create constexpr maps
* Feature: First version of GridProfile Parser which shows all values contained in the profile.
* webapp: Update dependencies
* Apply better variable names
* Remove not required casts
* Add additional compiler flags to prevent errors
* Add const statement to several variables
* Replace NULL by nullptr
* Update bblanchon/ArduinoJson from 6.21.3 to 6.21.4
* Add const keyword to method parameters
* Add const keyword to methods
* Use references instead of pointers whenver possible
* Adjust member variable names in MqttSettings
* Adjust member variable names in NetworkSettings
* webapp: Update timezone database to latest version
* webapp: Beautify and unify form footers
* Feature: Allow setting of an inverter limit of 0% and 0W
Thanks to @madmartin in #1270
* Feature: Allow links in device profiles
These links will be shown on the hardware settings page.
* Doc: Added hint regarding HMS-xxxx-xT-NA inverters
* Feature: Added DeviceProfile for CASmo-DTU
Based on #1565
* Upgrade actions/upload-artifact from v3 to v4
* Upgrade actions/download-artifact from v3 to v4
* webapp: add app.js.gz
* Gridprofileparser: Added latest known values
Thanks to @stefan123t and @noone2k
* webapp: Fix lint errors
* Feature: Add DTU to Home Assistant Auto Discovery
This is based on PR 1365 from @CFenner with several fixes and optimizations
* Fix: Remove debug output as it floods the console
* Fix: Gridprofileparser: Add additional error handling if profile is unknown
* webapp: add app.js.gz
* Fix: Offset cache for "YieldDay" did not work correctly
* webapp: update dependencies
* webapp: add app.js.gz
* Fix: yarn.lock was outdated
* Fix: yarn build error
* Fix: Reset Yield day correction in combination with Zero Yield Day on Midnight lead to wrong values.
* Fix: Allow negative values in GridProfileParser
* Correct variable name
* Fix#1579: Static IP in Ethernet mode did not work correctly
* Feature: Added diagram to display
This is based on the idea of @Henrik-Ingenieur and was discussed in #1504
* webapp: update dependencies
* webapp: add app.js.gz
---------
Co-authored-by: Thomas Basler <thomas@familie-basler.net>
Co-authored-by: Pierre Kancir <pierre.kancir.emn@gmail.com>
* add more values to web app live view. this should add all interesting
values for the web app live view. those include important values and
values that change frequently.
* add more interesting JK BMS dummy messages: one has 0% SoC and an
alarm (discharge undervoltage) set. the other has the undertemperature
alarm set.
* add alarms and warnings to live view
* publish alarm and status bits through MQTT individually
* publish cell voltages to MQTT broker
* remove trailing spaces in BatteryStats class
* VE.Direct: return non-nullptr as a fallback
the changed return statement was supposed to return a shared_ptr to a
new and valid MPPT data struct as a fallback. however, it did return a
new shared_ptr that was initialized to nullptr.
* VE.Direct: make liveview total use total MPPT values
this change makes the call to VictronMppt.getData() obsolete, which in
turn will therefore not cause an error message on the console if
VE.Direct (MPPT) is not enabled. this change also takes care that once
multiple VE.Direct MPPT charge controllers are supported, the sums of
the respective total values are used in the web app totals.
* introduce VictronMpptClass
this solves a design issue where the loop() method of a static instance
of VeDirectMpptController, which is part of library code, is called as
part of the main loop() implementation. that is a problem because the
call to this loop() must be handled differently from all other calls:
the lib does not know whether or not the feature is enabled at all.
also, the instance would not be initialized when enabling the feature
during normal operation. that would even lead to a nullptr exception
since the pointer to the serial implementation is still uninitialized.
this new intermediate class is implemented with the support for multiple
Victron charge controllers in mind. adding support for more charge
controllers should be more viable than ever.
fixes#481.
related to #397#129.
* VE.Direct: move get.*AsString methods to respective structs
those structs, which hold the data to be translated into strings, know
best how to translate them. this change also simplifies access to those
translation, as no parameter must be handed to the respective methods:
they now act upon the data of the instance they are called for. adds
constness to those methods.
* VE.Direct: simplify and clean up get.*AsString methods
use a map, which is much easier to maintain and which reads much easier.
move the strings to flash memory to save RAM.
* DPL: use VictronMpptClass::getPowerOutputWatts method
remove redundant calculation of output power from DPL. consider
separation of concern: VictronMpptClass will provide the total solar
output power. the DPL shall not concern itself about how that value is
calculated and it certainly should be unaware about how many MPPT charge
controllers there actually are.
* VE.Direct: avoid shadowing struct member "P"
P was part of the base struct for both MPPT and SmartShunt controller.
however, P was also part of the SmartShunt controller data struct,
shadowing the member in the base struct.
since P has slightly different meaning in MPPT versus SmartShunt, and
since P is calculated for MPPT controllers but read from SmartShunts, P
now lives in both derived structs, but not in the base struct.
* VE.Direct: isDataValid(): avoid copying data structs
pass a const reference to the base class implementation of isDataValid()
rather than a copy of the whole struct.
* VE.Direct: unify logging of text events
* VE.Direct: stop processing text event if handled by base
in case the base class processed a text event, do not try to match it
against values that are only valid in the derived class -- none will
match.
* VE.Direct MPPT: manage data in a shared_ptr
instead of handing out a reference to a struct which is part of a class
instance that may disappear, e.g., on a config change, we now manage the
lifetime of said data structure using a shared_ptr and hand out copies
of that shared_ptr. this makes sure that users have a valid copy of the
data as long as they hold the shared_ptr.
* VE.Direct MPPT: implement getDataAgeMillis()
this works even if millis() wraps around.
* VE.Direct: process frame end event only for valid frames
save a parameters, save a level of indention, save a function call for
invalid frames.
avoid staleness in case the same power limit is calculated over and over
again, hence no new power limit value is calculated and hence no power
limit command is sent to the inverter. staleness occurs in this case if
the first power limit command to establish the respective limit was not
received by the inverter. one can easily simulate a situation where the
same power limit is caluclated over and over again: with a battery above
the start threshold, set a very low upper power limit for the inverter
(DPL setting). that value will be used as the limit as long as the power
meter reading is larger than that.
we could also check the limit reported by the inverter. however, that
value is in percent of the inverter's max AC output, and is often not
the same value as we requested as the limit, but slightly off. we then
would have to decide how much deviation is okay, which is unreasonably
complicated.
closes#478.
Sunrise and -set must recomputed if one of the following conditions is met:
* The date changed (based on the selected timezone)
* Location (Lat/Lon) changed
* Sunset type changed
So instead of calculating that every minute just do it on update via web interface or date change.
If a new config is uploaded, the DTU gets restarted. There is no need to initiate a recalculation in this case.
* VE.Direct MPPT MQTT: remove trailing whitespace
* VE.Direct MPPT MQTT: publish P, IPV and E to MQTT
those values are calculated by OpenDTU-OnBatery and are part of the web
application live view, but were previously not published through MQTT.
closes#376.
* Move Mppt logic to subclass
* Added Definitions for Shunts and restructering
* First integration of SmartShunt data into Web Interface
* Code cleanup
* VE.Direct: whitespace cleanup
* VE.Direct: manage HardwareSerial in unique_ptr
* VE.Direct: _efficiency is only needed by MPPT
* VE.Direct: keep as many members private as possible
* VE.Direct: use int8_t for pins (as before)
* VictronSmartShunt: _verboseLogging is not used
* VE.Direct: OR (off reason) is MPPT specific
it also applies to Phoenix inverters and Smart BuckBoost, but since
there is no support for those, the code is moved to the MPPT controller.
* Added Shunt alarms to liveview
Changed from double to int for several readings
* Update build.yml to allow manual builds
---------
Co-authored-by: Philipp Sandhaus <philipp.sandhaus@cewe.de>
Co-authored-by: Bernhard Kirchen <schlimmchen@posteo.net>
for the MQTT integration, JkBms::DataPointContainer::updateFrom() was
changed such that the data points timestamp reflects the last change of
the data point value. that timestamp was used to set the SoC last update
timestamp. however, that timestampt must reflect the last time the SoC
was successfully received from the JK BMS so we could make sure the
value was up to date.
* JK BMS: avoid trailing whitespace in debug output
* JK BMS: publish data points through MQTT
* JK BMS: updateFrom: skip data points with equal value
this changes the interpretation of the timestamp in data containers that
are merely updated from other data containers: this is the oldest
timestamp known where the value was as recorded by the data point in its
respective container.
the data container constructed from an answer will -- naturally -- have
the timetamps of its data points set to the time they were constructed.
* JK BMS: only publish changed values to MQTT broker
all values are still published once every minute if the MQTT retain flag
is NOT set. otherwise, the constant values are only published once on
startup.
* JK BMS: avoid trailing whitespace in debug output
* JK BMS: publish data points through MQTT
* JK BMS: updateFrom: skip data points with equal value
this changes the interpretation of the timestamp in data containers that
are merely updated from other data containers: this is the oldest
timestamp known where the value was as recorded by the data point in its
respective container.
the data container constructed from an answer will -- naturally -- have
the timetamps of its data points set to the time they were constructed.
* JK BMS: only publish changed values to MQTT broker
all values are still published once every minute if the MQTT retain flag
is NOT set. otherwise, the constant values are only published once on
startup.
* thread-safety and dynamic memory for MessageOutput
* use dynamic memory to allow handling of arbitrary message lenghts.
* keep a message buffer for every task so no task ever mangles the
message of another task.
* every complete line is written to the serial console and moved to
a line buffer for sending them through the websocket.
* the websocket is always fed complete lines.
* make sure to feed only as many lines as possible to the websocket
handler, so that no lines are dropped.
* lock all MessageOutput state against concurrent access.
* MessageOutput: respect HardwareSerial buffer size
the MessageOutput class buffers whole lines of output printed by any
task in order to avoid mangling of text. that means we hand over full
lines to the HardwareSerial instance, which might be too much in one
call to write(buffer, size). we now check the return value of
write(buffer, size) and call the function again with the part of the
message that could not yet be written by HardwareSerial.
* VE.Direct: reset state machine on timeout
there will never be a large gap between two bytes of the same frame.
if such a large gap is observed, reset the state machine so it tries
to decode a new frame once more data arrives.
this is helpful in case of corrupted data that prevents the state
machine of transitioning to the final state even though the VE.Direct
data producer is done sending bytes that belong to the same frame.
* VE.Direct: print problems to MessageOutput
this includes the web console in particular, where many users have
access to while the serial console is not attached or monitored.
* VE.Direct: collect serial input into buffer and print
should help debug issues for users.
* VE.Direct: implement and use verbose logging switch
* add Icons for Battery and Victron device sensors in Home Assistant
overriding the boring default icon for many sensors
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* fix: homeassistant autodiscovery topics
homeassistant autodiscovery topics contain not allowed characters, which
are now fixed.
Sidenote: when autodiscovery messages were retained, the badly formatted messages
must be removed from the mqtt server manually. Otherwise the error
messages in homeassistant will persist.
Closes: https://github.com/helgeerbe/OpenDTU-OnBattery/issues/378
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* Fix: some homeassistant autoconf messages are wrong
Misc fixes for HA autoconfiguration:
* Entity sensor.panel_yield_today now uses state class "total"
* Entity sensor.panel_yield_yesterday now uses state class "total"
* Entity sensor.mppt_day_sequence_number_0364 now shows correct value
* sensor.panel_power duplicate removed
Closes: https://github.com/helgeerbe/OpenDTU-OnBattery/issues/379
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
---------
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* DPL MQTT handler: modernize
* there is no need to tokenize and check the topic of a received MQTT
message if we only subscribe to a single topic. all messages will be
for that topic. avoid testing the topic in the callback alltogether.
* use std::string and std::stoi over allocating and deleting a buffer
and copying charactes around.
* use a switch statement to process the actual payload.
* break a long line.
* DPL: fix getMode() return value
getMode() returned a bool. probably its return type was not adjusted
when the third mode was introduced. this lead to mode 2 being cast to
true implicitly, which in turn was used to construct a String, such that
"1" was published as the DPL mode when in fact it was 2.
make the mode an enum class to avoid such problems in the future.
inline getMode() and setMode().
fix indention.
the size allocated for the HTTP request response was too little, while
the size for the buffer of the websocket output was increased already.
add a new member variable and use it in both context, such that
increasing the buffer size to accomodate more space (for the JSON data
in particular) will benefit both contexts in the future.
* DPL: improve verbose logging
* shorten DPL log prefix
* canUseDirectSolarPower() was printed two times
* _batteryDischargeEnabled was printed two times
* convert boolean values to human-readable strings
* add units where possible
* split messages into block "before calculating new limit" and "after
calculating new limit", as the latter cannot rely on _inverter being
available.
* order messages such that variables whose value is derived from other
variables are printed later than their dependencies.
* merge output into blocks (one instance near "Printout some stats")
* remove more redundant info (produced in functions outside loop())
* print target grid consumption
* DPL: inhibit solar passthrough while stop threshold reached
* DPL: implement and use isBelowStopThreshold()
we only want to inhibit solar passthrough if the SoC is *below* the stop
threshold, not if it is equal to the stop threshold. otherwise, when
discharging, we would discharge until the battery reached the stop
threshold, then we would also inhibit solar passthrough, until the
battery is charged to the SoC stop threshold plus one percent.
processing a published valued on a subscribed topic is currently running
in a task that is not the task executing the main loop(). that's because
the espMqttClient(Secure) was constructed without arguments, which
selects the constructor with two arguments priority and core, both of
which have default values. that constructor selects
espMqttClientTypes::UseInternalTask::YES, causing a task to be created
in which context the MQTT client loop is executed.
MQTT subscribers assume they are running in the same context as the main
loop(). most code assumes exactly that. as the scheduler is preemptive
and very little (none at all?) code is interlocked, we have to make sure
to meet the programmer's expectations.
this changeset calls the MQTT client loop in the context of the main
loop() and enforces the use of espMqttClientTypes::UseInternalTask::NO.
* MQTT verbose logging: fix typo
this typo caused that verbose logging was always disabled for MQTT and
could not be enabled.
* webapp status: show MQTT verbose logging setting
* PowerMeter: gracefully handle non-float MQTT values
* PowerMeter: update _lastPowerMeterUpdate conservatively
update the timestampt only if the topic actually matched any
subscription and if the value could be parsed as a float.
* PowerMeter: unsubscribe before subscribing
* PowerMeter: organize subscriptions in a map
this allows for a slightly more elegant code and reduced amount of code
overall.
* PowerMeter: clean up header
* move private methods to private section of class declaration.
* remove unused member variable.
Make the administrative accesspoint timeout configurable. The default
value is 3 minutes, values from 0-99999 are possible, where 0 means
infinite (no timeout).
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* DPL: implement verbose logging switch
* MQTT: implement verbose logging switch
* power meter: implement verbose logging switch
* Hoymiles lib: implement verbose logging switch
* cpp linting: "final" makes "virtual" and "override" redundant
... however, using only "final" is not as verbose.
When OpenDTU has a Pylontech CAN Bus Battery connected and enabled, this
patch adds the discovery routine for Home Assistant
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
if the new calculated power limit is below the minimum power limit
setting, the inverter is shut down. the shutdown() function is called
every time this condition is detected, which is also true if the
inverter is kept shut down for longer. that happens while the battery
is charging in particular (solar passthrough off). there are other
cases.
in such cases we still want to get into the DPL status "stable". to be
able to determine this stable state, we must know if the call to
shutdown did actually initiate a shutdown or if the inverter is already
shut down.
we then can forward this "changed" or "not changed" info up the call
chain, where the loop() will know that the system is actually stable.
* fix another fixable "passtrough" typo
the typo in the config's identifier is not changed to preserve
compatibility while not spending the effort to migrate the setting.
* webapp language: prefer SoC over SOC
* DPL: implement solar passthrough loss factor
in (full) solar passthrough mode, the inverter output power is coupled
to the charge controler output power. the inverter efficiency is already
accounted for. however, the battery might still be slowly discharged for
two reasons: (1) line losses are not accounted for and (2) the inverter
outputs a little bit more than permitted by the power limit.
this is undesirable since the battery is significantly drained if solar
passthrough is active for a longer period of time. also, when using full
solar passthrough and a battery communication interface, the SoC will
slowly degrade to a value below the threshold value for full solar
passthrough. this makes the system switch from charging the battery
(potentially rapidly) to discharging the battery slowly. this switch
might happen in rather fast succession. that's effectively
trickle-charging the battery.
instead, this new factor helps to account for line losses between the
solar charge controller and the inverter, such that the battery is
actually not involved in solar passthrough. the value can be increased
until it is observed that the battery is not discharging when solar
passthrough is active.
In Home Assistant, when Home Assistant MQTT-Auto-Discovery is active,
almost all Sensors of the auto-discovered Victron device in Home
Assistant become "unavailable" after a short time - except those
Sensors with frequent changes like battery voltage or panel voltage.
This patch introduces regular mqtt updates for all VE.Direct sensors
when MQTT-Auto-Discovery is enabled.
Signed-off-by: Martin Dummer <martin.dummer@gmx.net>
* DPL: wait for valid time information
we know that the Hoymiles library refuses to send any message to any
inverter until the system has valid time information. until then we can
do nothing, not even shutdown the inverter.
* DPL: wait for device info to be ready
a calculated power limit will always be limited to the reported
device's max power. that upper limit is only known after the first
DevInfoSimpleCommand succeeded. wait for that information to be
available.
* DPL: fix initial calculcation backoff
if the calculation backoff is initialized to zero, the backoff will be
doubled to zero until a new, different power limit was calculated for
the first time. this lead to the DPL recalculating a power limit
hundreds of times without a backoff after startup.
* VE.Direct: remove polling interval
the polling interval was meant to limit the amount of MQTT updates.
however, that is already controlled by the global MQTT publish interval.
the removed interval was instead used to limit polling of the VE.Direct
UART for incoming data.
the Victron device sends data unsolicited. the VeDirectFrameHandler does
not implement any polling mechanism. no data is ever sent to the Victron
device.
what the removed polling interval did was cause a buffer overrun of the
HardwareSerial class, since the incoming data was not processed in time.
so every five seconds, we read a whole valid VE.Direct frame, plus some
old data, which was not a whole frame, leading to VE.Direct error
messages to pop up.
with the polling interval removed, no framing errors are reported, and
instead we gain new data from the charge controller approximately ever
two seconds -- for free.
* VE.Direct: change texts to correct VE.Direct capital letters
* VE.Direct: improve "UpdatesOnly" switch labels
especially since the publish interval setting is gone, the label makes
it hard to comprehend what the switch does. update the texts to better
explain what the switch is used for.
use the same text on the VE.Direct info view.
* VE.Direct: use StatusBadge on info view
there were custom badges to indicate the VE.Direct settings. replace
those by the common StatusBadge to make then look the same as the other
badged on the info views.
the requested newPowerLimit was indeed limited to the configured maximum
inverter output and the result was stored in effPowerLimit. later,
however, effPowerLimit must be scaled up (if necessary), not
newPowerLimit. the latter variable is not respecting the configured
maximum inverter output.
* DPL: use VeDirect.isDataValid()
in case the communication to the Victron charger is disrupted, the
respective values in VeDirect.veFrame are not invalidated such that we
would notice a problem. instead, isDataValid() should be used to make
sure that the Victron charger is actually alive and that we can trust to
use the reported values.
* DPL: simplify canUseDirectSolarPower return statement
a new status is needed to communicate that no update was sent to the
inverter because its power limit is still valid. in this case,
calculating a new power limit is delayed by an exponentially increasing
backoff. the maximum backoff time is ~1s, which is still plenty fast.
the backoff is actually necessary for another reason: at least
currently, a lot of debug messages are printed to the console. printing
all that information in every DPL loop() is too much.
the unconditional solar passthrough mode, configured using MQTT, works
differently than the normal mode of operation. it is also independent
from the power meter reading. if this mode is active, a shortcut is
taken to a function that implements the actions for this mode. this is
convenient since we don't have to consider special cases in the code
that handles normal mode of operation.
the DPL already took care to shut down the inverter if anything fishy
was going on, mainly to make sure that the battery is not drained.
however, some cases were missed:
* if the configuration changed such that another inverter is now
targeted, the one the DPL controlled previously was not shut down.
* if the configuration changed such that another inverter (different
serial number) was configured at the same index, the previous one
was not shut down.
this change corrects these problems by making the DPL keep a copy of the
shared_ptr to the inverter. the shared_ptr is only released once the DPL
shut the respective inverter down.
this implementation checks all requirements for a new power limit to be
calculated, one after the other. if any requirement is not met, a
respective status is announced.
status messages are communicated on the (serial) console. these can also
be displayed easily on the web app in the future. the status texts
explain clearly what the DPL is currently doing, which aids
understanding how the DPL works. the status is only announced if it
changes, or after a fixed interval.
as each requirement is checked individually, the code readability is
improved as well. previously, all the respective conditions had to be
checked as well, but the statements were more complex.
the DPL loop is now executed with high frequency, i.e., it does not wait
for a fixed timespan to pass before checking requirements. it always
aborts on the first unmet requirement. this should improve responsiveness,
as the DPL checks all requirements more often.
the DPL now waits for all power commands and power limit updates to
complete. when that is the case, a settling time elapses. after the
settling phase, the DPL waits for a new update from the inverter and
from the power meter. now it can be assumed that the values are in sync.
it then makes sense to calculate a new power limit immediately, which
the DPL then does.
