* show battery voltage, current, and power in live view header (the "totals")
* show battery current and power in extra card
* use soc and current precision in live view
* BatteryStats: do not knowingly publish invalid data: not all battery
providers know all values the base class manages. make sure to
prevent publishing invalid values.
Co-authored-by: Bernhard Kirchen <schlimmchen@posteo.net>
This breaks existing HASS automation, as entity names and MQTT topics change!
* include dtu-unique-id in DPL MQTT HASS topics to allow multiple DTUs to be controlled
* fix filename "src/MqttHandlVedirectHass.cpp"
* refactor: use values from 'MqttHandleHass', add 'via_device' to all HASS devices
* set step size for power limiter voltage values
Hoymiles inverters allow setting relative limits with a precision of 0.1 %.
this changeset allows to utilize this precision.
* preserve accuracy when decoding power limit
* Web API: process floating point limits
* MQTT: process floating point limits
* use appropriate accuracy for limits in web UI
* HASS: step for relative inverter limit is 0.1 %
make the MQTT power meter pre-selected when editing power meter settings
for the first time. the MQTT power meter is the most efficient and hence
preferred power meter implementation.
when performing a test request using the web UI, we need to init() the
respective power meter, but we do not want to start the polling task.
hence we move initialization of the polling task to the poll() function.
it will return if the task is setup already, otherwise setup the task.
implement a function which allows to reset the SML decoder. this new
function is used after a datagram ends. for the SML HTTP power meter
this is simple: after all bytes from the request's answer have been
decoded, we reset the decoder. for the SML serial power meter, we
perform the reset after a datagram ended based on timing (no new bytes
have been received for a specific amount of time).
cache the values decoded in the SML datagram and only copy them to the
local stash of values if the checksum of the SML datagram matched. also
makes sure that values from incomplete SML datagrams are not used.
moreover, we now only publish values to the MQTT broker that we actually
decoded (successfully) from an SML datagram (we previously published 0.0
as values to topics we never decoded a value for).
* when printing a message, tell the name of the derived class.
* print total power only when state SML_FINAL reached.
* tell if a checksum verification error occurred.
a 9600 baud serial interface does not need a hardware UART. these
changes switch the SDM power meter implementation to use a software
serial instance instead. this is desirable as hardware UARTs are scarce
and users need them for JK BMS and VE.Direct interfaces.
the destructor will block for way too long if we keep holding the
polling mutex while performing a transcation with the SDM power meter.
when reading, we now release the lock. afterwards, i.e., in between
transactions, we check the stop flag so the task terminates in a timely
manner once asked to do so.
the MQTT power meter can now process the messages published at the
respective topics as JSON and extract a power value using a JSON path
(same as in HTTP+JSON power meter). additionally, selecting a unit for
the power value as well as an option to invert the value's sign was
added as well, similar to the HTTPS+JSON power meter.
instead of reading the main config's powermeter struct(s), the
individual power meters now are instanciated using a copy of their
respective config. this allows to instanciate different power meters
with different configs. as a first step, this simplifies instanciating
power meters for test purposes.
all power meter providers now have their own configuration struct
defined. a respective method to serialize and deserialize the provider
config is implemented for each provider.
this new class uses the newly introduced HttpRequestConfig and performs
HTTP requests using this config. it will be reused for other power
meters (SML over HTTP(S)) and may be reused by other features in the
future (battery provider, solar power provider, etc.).
the extractUrlComponents method did extract username and password
from the URL and encoded it for basic authentication. however, the
respective result string was never used. we only perform basic
authentication if the auth type is "basic" and if username and
password were supplied through the respective inputs.
the parameters to peform an HTTP request by the HTTP(S)+JSON power meter
have been generalized by introducing a new config struct. this is now
used for all values which the HTTP(S)+JSON power meter can retrieve, and
also used by the HTTP+SML power meter implementation. we anticipate that
other feature will use this config as well.
generalizing also allows to share serialization and deserialization
methods in the configuration handler and the web API handler, leading to
de-duplication of code and reduced flash memory usage.
a new web UI component is implemented to manage a set of HTTP request
settings.
avoid additional conversions and avoid double for the fact that
calculations on type double are implemented in software, whereas
float is handled in hardware on ESP32.
this new class handles SML data. it uses the SML lib to decode values
and manages those. this de-duplicates code as the class is applicable
to all power meters that collect SML data.
this setting was not used. the baud rate for the SDM is set to 9600 in
the source code. until the baud rate being customizable is actually
required by somebody, we remove the setting altogether.
"powertotal" is always published and it is published by the base class
directly. other values are still published by the derived classes, but
use a base class method, which takes care that a common base topic is
used in particular.
instead of iterating a map with subscriptions, we now bind the target
variable to the callback, which is executed once a message is arrived.
this way, the target variable is already linked to the respective topic
when the callback is executed.
lock the mutex when writing the variable, as the MQTT callback is
executed in a different context (MQTT task) than the main loop task,
which otherwise accesses the variables.
it is important to separate the capabilities of each power meter
provider into their own class/source file, as the providers work
fundamentally different and their implementations must not be
intermangled, which made maintenance and improvements a nightmare
in the past.
this only changes line endings. inspect this commit with command `git
show <commit-sha> --ignore-space-at-eol` and it will tell you that the
commit appears to be "empty" (since all changes are whitespace changes
near the end of a line, which are ignored in that git show command).
the files to be changed were found and updated using this command:
find lib src include webapp/src -type f | \
xargs grep --binary-files=without-match --files-with-matches \
$(printf '\r\n') | xargs dos2unix
the following files were restored afterwards, as they are using CRLF
line endings in the upstream as well:
- lib/CMT2300a/cmt2300a_defs.h
- lib/README
- include/README
broke in be41e6b9. was unusable as the complete type of DummySerial has
to be available in the JK BMS controller header when defining the
unique_ptr managing the dummy instance. this problem is solved by moving
the whole dummy class into its own header.
instead of hard-coding the use of hardware UART 2, the SDM power meter
instance now asks for a free hardware serial port to use and
instanciates the respective HardwareSerial object using said port.
get rid of particular compile-time designations by UART index. just hand
out the next free index of hardware UARTs, or indicate that none is
available any more.
use names as keys to register and free UARTs.
only on ESP32-S3-USB. this fiddles with the available hardware UARTs to
make it possible to use a third Victron MPPT. if three MPPTs are defined
int the pin mapping, you will not be able to use the SmartShunt and JK
BMS battery interfaces.
note that using a second MPPT will also conflict with the SDM power
meter, and that conflict is not detected, yet.
we used this library solely to interpret the answer of an HTTP web
server as JSON and find a particular value using a path expression in
the HTTP power meter implementation.
since we ran out of flash memory on non-S3 ESP32, we need to cut some
corners. removing FirebaseJson is the last low-hanging fruit that we
currently know of. we can get rid of it by using ArduinoJson (which is
already integral part of the firmware) and implementing a custom logic
to extract a value based on a path expression.
other than the FirebaseJson path "finder", the new implementation
only knows how to access sub-keys delimited by a forward slash. in
particular, accessing array members is not supported any more. I am
hoping that this is simply not an issue. if so, we will have users
complaining and we can add this functionality in a later release.
merge upstream tag v24.4.12, resolve conflicts (helgeerbe), fix eslint errors (schlimmchen) and adopt new web api method to save code duplication (schlimmchen).
this allows to use two VE.Direct interfaces, as there is no conflict
regarding HW serial port 2 after making the battery interfaces use
serial port 0 on devices with USB CDC. on those chips HW serial 0 is
free to be used since serial messages are written through the USB
interface directly.
we found that the inverter sometimes stops responding to commands,
especially to the "start producing" command. we now count the number of
consecutive timeouts when trying to send a new limit or power state
commands. after two timeouts were recorded, every additional timeout
will send a restart command to the inverter.
as a last resort, if the counter keeps climbing, the DTU is restarted.
notice that this only targets unresponsive inverters which are
reachable. unreachable inverters are not restarted and do not cause a
DTU reboot. this is important for solar-driven inverters, which are
unreachable during the night. the DPL will not calculate a new limit and
hence the updateInverter() method will do nothing while the target
inverter is unreachable.
publish the timeout counter to MQTT for monitoring purposes.
avoid performing a calculation based on a (slightly) outdated power
meter reading, which was aquired just before the limit was actually
applied by the inverter, but which was received by OpenDTU-OnBattery
after the inverter stats.
without a power meter configured, the DPL now sets the base load as the
inverter limit if the battery charge allows it. it also takes
solar-passthrough into account, i.e., if the battery is in a charge
cycle but the solar output (Victron MPPT) is significant, the solar
power will be used up until the base load. if the battery reaches the
full solar passthrough threshold, the DPL will match the inverter limit
to the MPPT solar output.
on power meter issues (usually a timeout), keep the inverter enabled and
make it produce the configured base load limit if the battery can be
discharged. that should be okay since the base load config value is
expected to be small and a little less than the actual household base
load, i.e., if this amount of power is produced, the household will
consume it in any case and no energy is fed into the grid.
double precision floating point numbers are not needed to handle
VE.Direct values. handling double is implemented in software and hence
*much* more resource intensive.
queue every text event until the frame was checked by it checksum. then
process the data directly into the buffer struct. do not clear the
buffer struct, so it will always include the most recent value of a
particular data point.
users are manipulating the DPL using HTTP POST requests. often they are
requesting the current settings using HTTP GET on the respective route,
then change a particular settings, and send all the data back using HTTP
POST. if they failed to remove the metadata node from the JSON,
OpenDTU-OnBattery would not be able to process the JSON due to its size.
the web app does not submit the metadata.
to avoid problems, the metadata is now split from the configuration
data.
this method calls the overflowed() method on the respective
DynamicJsonDocument and prints a respective message if not all
data could be added to the DynamicJsonDocument.
* updating the SoC or value shall also update the general timestamp, as
the latter is defined as "any value changed", which includes SoC and
voltage, of course.
* if the last update is not a valid timestamp at all, the
updateAvailable method must always return false, obviously.
determine the amount of controllers actually in use dynamically,
especially to avoid indices which are invalid, causing an error
to be printed, even though the user did not do anything wrong.
in your pin_mapping.json, add a powermeter object like this:
[
{
"name": "My Board",
...
"powermeter": {
"rx": <num>,
"tx": <num>,
"dere": <num>
},
...
}
]
the SML power meter requires the rx pin to be set. the SDM power meter
requires the rx and tx pins are set. the "dere" pin pin is optional and
if set, this pin controls the driver enable and receiver enable pins of
the RS485 transceiver. the SDM library handles this pin.
closes#771.
* remove/comment unused variables to avoid compiler warnings
* cleanups: fix indention and style, make variable private, implement
getters in header and make const.
* optimize message output: respect verbose logging setting. prefix
output with SMA_HM.
* use newly introduced mutex in PowerMeterClass also for SMA HomeManager
* refactor code for readibility, unindent where possible.
the SDM power meter (among others) writes the power consumption of three
phases in multiple steps. this change helps to prevent getTotalPower()
reading intermediate values, e.g., reading a new value for phase 1 but
old values for phase 2 and 3 since phase 2 is currently read.
cache the values, and write them all at once, protected by a mutex,
later.
closes#732.
this changeset refactors the web application's DPL settings view. the
DPL settings can be complex, and they shall be presented in a way that
allows users to comprehend their meaning. irrelevant settings are now
hidden or displayed dynamically based on the influencing settings.
* group SoC thresholds into their own card
* hide battery SoC thresholds if battery disabled. if the user did not
even enable the battery interface, battery SoC values will not be used
for DPL decisions. in that case we completely hide the respective
settings from the DPL admin view. this reduces the amount of settings
for new users and especially users who don't even have a battery in
their setup or have no BMS connected.
* group voltage thresholds and improve label texts
* fix load correction factor unit
* fix header (wording)
* group solar-passthrough settings in new card
* group inverter-related settings
* hide solar passthrough settings if VE.Direct is disabled. closes#662.
* completely disable form if any requirement is not met
* list available inverters by name and type. this makes it much more
convenient to select the right inverter, especially since the order of
the inverters in the web UI is decoupled from their position in the
internal array, which was used to select them previously. care was
taken that old configs select the same inverter after an update.
when editing the DPL settings, the selects an inverter from the newly
created drow-down list, and the respective old inverter is
pre-selected.
* disable form if no inverter is configured (config alert)
* make inverter input selection dynamic. adjust selection to actual
amount of channels for selected inverter. skip selection altogether if
inverter has only one channel, or if it is solar powered.
* web app: wording adjustments
* group meta data into new property and exclude from submission. saves
memory when evaluating the submitted settings.
* hide irrelevant settings if inverter is solar-powered
* move restart hour setting to inverter card. translate setting which
disabled automatic restart.
* simplify "drain strategy" setting into an on/off toggle. care was
taken that existing configs work the same after an upgrade. the
respective drain strategy is translated into the new setting when
reading the config. once the config is written, the new setting is
persisted and the old is not part of the config any more.
* show more configuration hints, depending on actual configuration
* replace inputs by InputElement components where possible
* fix compiler warning in SerialPortManager.cpp: function must not
return void
* clean up and simplify implementation of usesHwPort2()
* make const
* overrides are final
* default implementation returns false
* implement in header, as the implementation is very simple
* rename PortManager to SerialPortManager. as "PortManager" is too
generic, the static instance of the serial port manager is renamed to
"SerialPortManager". the class is therefore renamed to
SerialPortManagerClass, which is in line with other (static) classes
withing OpenDTU(-OnBattery).
* implement separate data ages for MPPT charge controllers
* make sure MPPT data and live data time out
* do not use invalid data of MPPT controlers for calculations
* add :key binding to v-for iterating over MPPT instances
