If the string contains control characters for some reason, the browser
will reject the json with the error `bad control character in string
literal`.
This adds a setManufacturer function that validates the string is ASCII
and will cut off the string at the first non-ascii character.
Pylontech: `PYLON` (50 59 4C 4F 4E 20 20 20)
Pytes: `PYTES` (50 59 54 45 53)
Deye: `DY001` (44 59 30 30 31 03 E8 03)
See https://github.com/helgeerbe/OpenDTU-OnBattery/discussions/1226#discussioncomment-10566898
Commit accc70dea0 added the battery SoC to
the live view header. But due to getSoC() returning an int,
the precision was limited.
This changes getSoC() to return float so when a source with higher
precision is available, the respective precision is shown.
we are running out of level 1 interrupts on ESP32-S3 boards. for that
reason, until this prooves to be another sort of problem, we allocate
the TWAI's interrupt for the battery CAN implementation at level 2.
the stack size was already increased by Andreas Boehm in df53f34b51 in
the context of #1144 (SPI port manager). this change aligns the stack
size to a power of two and adds comments. the commit also serves to
place this change more prominently as a fix in the changelogs.
this allows us to add a valid Authorization header to each but the first
GET request, saving us from performing two GET requests every time we
want to perform the GET request. we still need a new client nonce and we
need to increase the nonce counter, so we also need to calculate a whole
new response, as we cannot just reuse the previous Authorization header
(that would be a replay attack).
the MD5 scheme should still be widely deployed, even though it is
deprecated. it is also still the default if not specific algorithm
is requested by the server.
we previously performed a whole new GET request when doing digest
authentication. it seemed beneficial to reuse the TCP connection to
perform the second GET request, which includes the authentication
tokens. however, if the server sends "Connection: close" we must not
requse the TCP connection for another HTTP request.
this broke authentication against Shelly devices (at least those with
original firmware).
now we explicitly set "Connection: keep-alive" in our request, and reuse
the TCP connection only if te server replies with "Connection:
keep-alive" as well.
the SMA power meter implementation did not announce that it received a
new datum, such that the power meter data age was never reset. this made
the power meter values outdated and hence invalid, even though new
values were received and processed.
the original implementation of the isDataValid() method worked by
returning false if any of the charge controllers had invalid data. if no
charge controllers were configured, this function then also needed to
also return false.
the logic was changed in 415c767d such that if at least one charge
controller had valid data, the function would return true. this would
have required to adjust the default return statement as well, but it was
not.
if the user enabled VE.Diret and configured at least one charge
controller in the pin mapping, but this controller never delivered any
data, the function would now errorneously return true, because the
container is not empty.
however, if no controller has valid data, this now means we exit the
loop and then the return value must be false, which is also desired if
the container is empty.
* update mqtt subscriptions when topic was changed
* DPL/Huawei: manage MQTT subscriptions in map
---------
Co-authored-by: Bernhard Kirchen <schlimmchen@posteo.net>
previously, we needed VE.Direct enabled to know the
solar charge controller output, which solely decided when
it was "nighttime". since this is now determined by the
wall clock, we can offer this feature to users without a
Victron charge controller connected using VE.Direct.
* backport SPIPortManager from @skippermeister
* adapt to support ESP32 and ESP32-S3 and ESP32-C3
* use logic to work with SPI numbering as in the official
documentation: start with SPI0 and go up to SPI3
* increase Huawei CAN controller stack size to 2000
* increase startup delay for USB_CDC enabled builds to be able to
catch bootlogs over USB
* fix: DPL: start discharging at night logic error
the switch "always start discharging battery at night" would cause to
stop discharging the battery when there was solar power and the battery
was discharged below the start threshold.
this change introduces a nighttime discharging boolean variable, which
is enabled the instant we decide to start a battery discharge cycle due
to nighttime havin arrived. we reset this variable as soon as it is
daytime (solar power available). in that case, we allow discharging the
battery if the start threshold was reached. this can actually be the
case if the battery is charged with cheap electricity during the night.
removed comments as they merely spell out what the if statement already
expresses quite nicely.
* use SunPosition.isDayPeriod() to check for daytime
---------
Co-authored-by: Andreas Böhm <andreas@boehm.cx>
this changeset adds support for parsing the MQTT battery provider's SoC
and voltage topics' payloads as JSON to extract a numeric value at a
configurable path.
* show battery voltage, current, and power in live view header (the "totals")
* show battery current and power in extra card
* use soc and current precision in live view
* BatteryStats: do not knowingly publish invalid data: not all battery
providers know all values the base class manages. make sure to
prevent publishing invalid values.
Co-authored-by: Bernhard Kirchen <schlimmchen@posteo.net>
This breaks existing HASS automation, as entity names and MQTT topics change!
* include dtu-unique-id in DPL MQTT HASS topics to allow multiple DTUs to be controlled
* fix filename "src/MqttHandlVedirectHass.cpp"
* refactor: use values from 'MqttHandleHass', add 'via_device' to all HASS devices
* set step size for power limiter voltage values
Hoymiles inverters allow setting relative limits with a precision of 0.1 %.
this changeset allows to utilize this precision.
* preserve accuracy when decoding power limit
* Web API: process floating point limits
* MQTT: process floating point limits
* use appropriate accuracy for limits in web UI
* HASS: step for relative inverter limit is 0.1 %
we previously used the mutex to protect writing the target variable.
however, we would only do that for the old usecase, where a plain float
value in Watts was expected as the topic's payload.
enabling the HTTP+JSON power meter without enabling at least one value
makes no sense, so value 1 has always been treated as "always enabled".
while doing the power meter refactoring, however, a regression was
introduced that would hide the settings for value 1 in the web UI until
the power meter settings would be saved, which they cannot be due to
missing settings for value 1.
when performing a test request using the web UI, we need to init() the
respective power meter, but we do not want to start the polling task.
hence we move initialization of the polling task to the poll() function.
it will return if the task is setup already, otherwise setup the task.
implement a function which allows to reset the SML decoder. this new
function is used after a datagram ends. for the SML HTTP power meter
this is simple: after all bytes from the request's answer have been
decoded, we reset the decoder. for the SML serial power meter, we
perform the reset after a datagram ended based on timing (no new bytes
have been received for a specific amount of time).
cache the values decoded in the SML datagram and only copy them to the
local stash of values if the checksum of the SML datagram matched. also
makes sure that values from incomplete SML datagrams are not used.
moreover, we now only publish values to the MQTT broker that we actually
decoded (successfully) from an SML datagram (we previously published 0.0
as values to topics we never decoded a value for).
* when printing a message, tell the name of the derived class.
* print total power only when state SML_FINAL reached.
* tell if a checksum verification error occurred.
a 9600 baud serial interface does not need a hardware UART. these
changes switch the SDM power meter implementation to use a software
serial instance instead. this is desirable as hardware UARTs are scarce
and users need them for JK BMS and VE.Direct interfaces.
the destructor will block for way too long if we keep holding the
polling mutex while performing a transcation with the SDM power meter.
when reading, we now release the lock. afterwards, i.e., in between
transactions, we check the stop flag so the task terminates in a timely
manner once asked to do so.
a 1-phase SDM power meter does not know about power or voltage of phase
2 or 3. do not publish values to the respective MQTT topics when using a
single phase SDM power meter.
avoid problems with chunked transfer encoding when using the client's
stream to parse a JSON document. fixes the HTTP+JSON power meter to work
with shelly and hichi (Tasmota).
the MQTT power meter can now process the messages published at the
respective topics as JSON and extract a power value using a JSON path
(same as in HTTP+JSON power meter). additionally, selecting a unit for
the power value as well as an option to invert the value's sign was
added as well, similar to the HTTPS+JSON power meter.
instead of reading the main config's powermeter struct(s), the
individual power meters now are instanciated using a copy of their
respective config. this allows to instanciate different power meters
with different configs. as a first step, this simplifies instanciating
power meters for test purposes.
all power meter providers now have their own configuration struct
defined. a respective method to serialize and deserialize the provider
config is implemented for each provider.
apply all config values from the webfrontend, then perform one polling
cycle. display values seperately in the result, and show the resulting
value as well.
this new class uses the newly introduced HttpRequestConfig and performs
HTTP requests using this config. it will be reused for other power
meters (SML over HTTP(S)) and may be reused by other features in the
future (battery provider, solar power provider, etc.).
the extractUrlComponents method did extract username and password
from the URL and encoded it for basic authentication. however, the
respective result string was never used. we only perform basic
authentication if the auth type is "basic" and if username and
password were supplied through the respective inputs.
the parameters to peform an HTTP request by the HTTP(S)+JSON power meter
have been generalized by introducing a new config struct. this is now
used for all values which the HTTP(S)+JSON power meter can retrieve, and
also used by the HTTP+SML power meter implementation. we anticipate that
other feature will use this config as well.
generalizing also allows to share serialization and deserialization
methods in the configuration handler and the web API handler, leading to
de-duplication of code and reduced flash memory usage.
a new web UI component is implemented to manage a set of HTTP request
settings.
avoid additional conversions and avoid double for the fact that
calculations on type double are implemented in software, whereas
float is handled in hardware on ESP32.
this new class handles SML data. it uses the SML lib to decode values
and manages those. this de-duplicates code as the class is applicable
to all power meters that collect SML data.
this setting was not used. the baud rate for the SDM is set to 9600 in
the source code. until the baud rate being customizable is actually
required by somebody, we remove the setting altogether.
"powertotal" is always published and it is published by the base class
directly. other values are still published by the derived classes, but
use a base class method, which takes care that a common base topic is
used in particular.
instead of iterating a map with subscriptions, we now bind the target
variable to the callback, which is executed once a message is arrived.
this way, the target variable is already linked to the respective topic
when the callback is executed.
lock the mutex when writing the variable, as the MQTT callback is
executed in a different context (MQTT task) than the main loop task,
which otherwise accesses the variables.
it is important to separate the capabilities of each power meter
provider into their own class/source file, as the providers work
fundamentally different and their implementations must not be
intermangled, which made maintenance and improvements a nightmare
in the past.
this only changes line endings. inspect this commit with command `git
show <commit-sha> --ignore-space-at-eol` and it will tell you that the
commit appears to be "empty" (since all changes are whitespace changes
near the end of a line, which are ignored in that git show command).
the files to be changed were found and updated using this command:
find lib src include webapp/src -type f | \
xargs grep --binary-files=without-match --files-with-matches \
$(printf '\r\n') | xargs dos2unix
the following files were restored afterwards, as they are using CRLF
line endings in the upstream as well:
- lib/CMT2300a/cmt2300a_defs.h
- lib/README
- include/README
the "serial console" over USB would be garbled badly after switching to
"platform = espressif32@6.7.0". this did not happen to the upstream
version of MessageOutput. we used Serial.flush(), which seemed to be
good in the respective context. however, the changes in
github.com/espressif/arduino-esp32/pull/9462
made flush() detrimental. we remove the use of flush(), as it seems not
to be required (in particular, the upstream project does not use it).
broke in be41e6b9. was unusable as the complete type of DummySerial has
to be available in the JK BMS controller header when defining the
unique_ptr managing the dummy instance. this problem is solved by moving
the whole dummy class into its own header.
instead of hard-coding the use of hardware UART 2, the SDM power meter
instance now asks for a free hardware serial port to use and
instanciates the respective HardwareSerial object using said port.
get rid of particular compile-time designations by UART index. just hand
out the next free index of hardware UARTs, or indicate that none is
available any more.
use names as keys to register and free UARTs.
only on ESP32-S3-USB. this fiddles with the available hardware UARTs to
make it possible to use a third Victron MPPT. if three MPPTs are defined
int the pin mapping, you will not be able to use the SmartShunt and JK
BMS battery interfaces.
note that using a second MPPT will also conflict with the SDM power
meter, and that conflict is not detected, yet.
