Commit Graph

875 Commits

Author SHA1 Message Date
Bernhard Kirchen
c2b49931be Fix: must call Pylontech Home Assistent init() method
previously, the Pytlontech Home Assistent class implementation had an
init() method, that was never called, as it did nothing. the class
relied on its loop() method being called from the main loop(). after
switching to the TaskScheduler approach, the Pylontech Home Assistent
class init() method was adjusted to register a task that calls the
loop() method periodically. however, the init() method was still not
called.
2024-02-05 20:54:34 +01:00
Thomas Basler
7e2064e264 Feature: Added support for boards with 16MB flash and PSRAM 2024-02-02 20:33:42 +01:00
Thomas Basler
1b637f0870 BREAKING CHANGE: Web API Endpoint /api/livedata/status and /api/prometheus/metrics
Yield total and Yield day where moved from the AC section to the INV section
2024-01-30 22:29:25 +01:00
Thomas Basler
e1564780d6 BREAKING CHANGE: Web API Endpoint /api/livedata/status and /api/prometheus/metrics
Power DC was moved from the AC section to the INV section
2024-01-30 22:29:25 +01:00
Thomas Basler
c27ecc3620 BREAKING CHANGE: Web API Endpoint /api/livedata/status
To reduce the heap usage it is necessary to send the inverters one by one instead of a huge response. A simple call to `/api/livedata/status` returns just some very general information. If detailed inverter information are required the inverter serial number has to appended `?inv=<serial number>`.
The websocket also returns only one inverter at a time. It as to be assembled at client side.
2024-01-30 22:29:25 +01:00
Thomas Basler
557c5d645e Remove all files but the pin_mapping.json from filesystem on factory reset
This allows to create more user defined files and get them deleted on factory reset.
2024-01-30 22:29:12 +01:00
Thomas Basler
48a722f826 Merge branch 'pr1642' into dev 2024-01-30 20:47:45 +01:00
Thomas Basler
a848275bb9 Merge branch 'pr1706' into dev 2024-01-30 19:34:19 +01:00
Sabouflage
5d7512e026 fix(mqtt): setting MQTT LWT online message for frontend 2024-01-30 19:03:23 +01:00
Sabouflage
dfed23261a fix(mqtt): MQTTs LWT QoS config used also for TLS connections 2024-01-30 19:02:57 +01:00
Stefan Oberhumer
e752c433af Use http header ETag caching for all static content.
Using the md5sum as ETag http header value should enable caching on all static http content.
2024-01-30 00:35:23 +01:00
Thomas Basler
2716f4c5df Removed not required pointer to AsyncServer instance 2024-01-27 01:09:55 +01:00
Thomas Basler
f0061b976d Merge branch 'pr1676' into dev 2024-01-26 16:48:15 +01:00
Thomas Basler
4e669d8932 Feature: Add support for ST7567 GM12864I-59N Display 2024-01-26 16:05:56 +01:00
helgeerbe
1916d6d6a8 check twice a second for new vedirect data on live viw 2024-01-26 14:32:44 +01:00
helgeerbe
ebacc2f25f Merge remote-tracking branch 'tbnobody/OpenDTU/master' into development 2024-01-25 23:36:32 +01:00
Stefan Oberhumer
7aece2e143 Call SunPosition.isDayPeriod() once 2024-01-23 09:42:13 +01:00
Thomas Basler
4f2fbaaf2a Remove unused function parameter 2024-01-20 23:12:59 +01:00
Thomas Basler
16fbad92ac simplify calculations in WebApi_ws_Live 2024-01-20 11:38:52 +01:00
Thomas Basler
7bc1a17fac Move task initialization from init method to constructor
This saves flash
2024-01-20 11:24:57 +01:00
Thomas Basler
251d197fb6 Migrate WebApi loop() methods to scheduler tasks 2024-01-20 02:00:22 +01:00
Thomas Basler
e66060e769 Move the conversation from time_t to String into DevInfoParser 2024-01-20 01:09:42 +01:00
Thomas Basler
dcc157261e Don't perform redundant conversions 2024-01-20 00:32:13 +01:00
helgeerbe
f305b40be0 Merge pull request #612 from Fribur:development
- fix to HttpPowermeter not using explicitly specified non standard ports
- Revert back to using FirebaseJson instead of ArduinoJson
2024-01-17 13:26:52 +01:00
helgeerbe
63205f88be Merge remote-tracking branch 'tbnobody/OpenDTU/master' into development 2024-01-16 17:39:45 +01:00
Thomas Basler
c20caf8097 Feature: Support HMS/HMT inverters in different countries with different frequency bands
Thanks to @Fribur, @homeautomation2022 and @stefan123t
2024-01-14 16:37:34 +01:00
Thomas Basler
2c92750100 webapp: Remove hard coded min/max frequencies for CMT module 2024-01-13 12:51:20 +01:00
Thomas Basler
ee78698e37 Migrate all frequency calculations to Hz
Previously the code contains calculations  using a mixture  of kHz and Hz.

Thanks to @Fribur
2024-01-13 12:51:20 +01:00
Thomas Basler
638f52a8da Feature: Add current limit to prometheus output
Fixes #1617
2024-01-13 11:33:58 +01:00
Thomas Basler
f013698471 Feature: Added fullscreen diagram to display 2024-01-13 11:31:12 +01:00
Fribur
e136e096f4
Merge branch 'development' into development 2024-01-08 11:00:59 -05:00
Thomas Basler
637d4f06a3 Feature: Added option to disable the diagram at the display 2024-01-08 14:19:26 +01:00
Bernhard Kirchen
a012d81427
avoid too frequent SmartShunt data copies (#596)
currently the whole SmartShunt data structure is copied to the
BatteryStats instance in every loop, even though the data cannot
possibly have changed. this is quite an expensive task to do in every
loop. this change tracks the last update timestamp and only does the
copy operation if an actual updated data structure was received from
the smart shunt.
2024-01-07 18:30:02 +01:00
helgeerbe
dd5d5ce9fd Prevent empty HASS auto discovery topics if memory allocation fails (onBattery) 2024-01-07 10:31:06 +01:00
helgeerbe
d0e1da7b1d Prevent config corruption by checking whether memory allocation was successfull. (onBattery) 2024-01-07 10:30:12 +01:00
helgeerbe
d6d274f078 Prevent empty HASS auto discovery topics if memory allocation fails (onBattery) 2024-01-06 22:51:35 +01:00
helgeerbe
ec93004724 Better handling of out of memory situations in live data websocket (onBattery) 2024-01-06 22:42:34 +01:00
helgeerbe
210fce67ce Use auto keyword and references more often (onBattery) 2024-01-06 22:10:07 +01:00
Bernhard Kirchen
2608080708 Feature: add heap details to system info and prometheus (#595)
this change adds the values of ESP.gteMaxAllocHeap() and
ESP.getMinFreeHead() to the prometheus metrics and the system
information object. the web UI uses these values to diplay the size of
the largest free contiguous block, calculate a rough estimate for the
level of fragmentation, and the maximum usage of heap memory since boot
in absolute and relative amounts.
2024-01-06 20:12:35 +01:00
Fribur
92de3e9f87 fixed a bug where under one condition DNS was not tried for resolving host IP 2024-01-05 20:54:53 -05:00
helgeerbe
d5155a07be Remove F macro from onBattery extensions 2024-01-05 23:02:26 +01:00
Thomas Basler
6573c51052 Use auto keyword and references more often 2024-01-05 22:04:54 +01:00
Thomas Basler
24c8a40fad Use Utils::checkJsonAlloc in ConfigurationClass 2024-01-05 21:56:34 +01:00
Thomas Basler
f968179b60 Better handling of out of memory situations in live data websocket 2024-01-05 21:56:34 +01:00
Thomas Basler
f00cd1bd61 Prevent empty HASS auto discovery topics if memory allocation fails 2024-01-05 21:55:15 +01:00
Thomas Basler
88d75673fc Prevent config corruption by checking whether memory allocation was successfull. 2024-01-05 21:55:15 +01:00
Thomas Basler
a65f1e48a5 Fix: Access Point not working after firmware update
Fixes #1613
2024-01-05 21:55:15 +01:00
Thomas Basler
2a15677923 Fix: Optimize network connection handling
This should provide a more reliable connection to several AP types. See #576
2024-01-05 21:55:15 +01:00
Bernhard Kirchen
377406f10c
Feature: add heap details to system info and prometheus (#595)
this change adds the values of ESP.gteMaxAllocHeap() and
ESP.getMinFreeHead() to the prometheus metrics and the system
information object. the web UI uses these values to diplay the size of
the largest free contiguous block, calculate a rough estimate for the
level of fragmentation, and the maximum usage of heap memory since boot
in absolute and relative amounts.
2024-01-05 21:46:31 +01:00
Fribur
e09ffcbb53 shorter parameter names 2024-01-05 14:39:32 -05:00
Fribur
85d0f2a8fb HttpPowerMeterClass: change order of resolving hostname
OpenDTU console gets spammed with "WifiGeneric::hostByName() error when first trying to resolve the hostname via DNS. So reverse order: first try mDNS, if that fails try DNS. Also ensure that https bool is passed correctly to HTTPClient::begin(). Lastly, concatenate strings for building Digest authorization using "+" and not via snprintf.
2024-01-05 14:36:19 -05:00
Fribur
9ed5a78818 Reverted changes to PowerLimiter, adapted DNS and mDNS handling in HttpPowerMeter
For non IP address URLs, HttpPowerMeter now first tries DNS for resolution as done in WifiClient::connect, and only if that fails tries mDNS. For the latter to work mDNS needs to be enabled in settings. Log in console if mDNS is disabled. String building for Digest authorization still tries to avoid "+" for reasons outlined in https://cpp4arduino.com/2020/02/07/how-to-format-strings-without-the-string-class.html This should also be saver than just concatenating user input strings in preventing format string attacks. https://owasp.org/www-community/attacks/Format_string_attack
2024-01-05 10:13:16 -05:00
Fribur
bc38ce344f remove FirebaseJson from platfromio.ini, fix unintended change in PowerLimiter 2024-01-04 18:22:58 -05:00
Bernhard Kirchen
3c8b8d4427
use frozen::string and frozen::map where reasonable (#593)
this change utilizes some of the features from library "frozen", which
was included upstream for the grid profile parser. to improve code
maintainability, a couple of std::maps mapping strings to values or the
other way around were introduced in OpenDTU-OnBattery-specific code at
the expense of some flash and computing overhead.

library "frozen" offers constexpr versions of map and string, which
saves initialization code and offers slightly faster lookups. this
brings the binary size down by ~25kB and should provide a small
performance improvement at runtime.
2024-01-04 23:28:34 +01:00
Fribur
d5eba2392c fixed long/float parsing bug 2024-01-04 16:32:42 -05:00
Fribur
f5c69060f5 re-factoring of HttpPowerMeter
Added ability to deal with local host names (mDNS), remove use of FirebasedJson to save ~20kB build size, some changes to PowerLimiter to avoid setting new inverter power limits when not needed (=current limit as reported by inverter is within hysteresis)
2024-01-04 16:20:32 -05:00
Bernhard Kirchen
e7a005839b
Feature: implement MQTT-driven battery provider (#589)
this battery provider implementation subscribes to a user-configurable
MQTT topic to retrieve the battery SoC value. the value is not
re-published under a different topic. there is no card created in the
web app's live view, since the SoC is already part of the totals at the
top of the live view. that is the only info this battery provider
implements.

closes #293.
relates to #581.
2024-01-04 15:42:10 +01:00
Thomas Basler
3c37b61f44 Use auto keyword and references more often 2024-01-04 15:38:56 +01:00
helgeerbe
65319ed07e log if memory allocation for live view fails
Explained in #591
2024-01-04 15:38:28 +01:00
Thomas Basler
b0b9764a23 Use Utils::checkJsonAlloc in ConfigurationClass 2024-01-04 14:47:21 +01:00
Thomas Basler
ef9f0040e8 Better handling of out of memory situations in live data websocket 2024-01-04 14:47:21 +01:00
Thomas Basler
ca18d2c841 Prevent empty HASS auto discovery topics if memory allocation fails 2024-01-04 14:09:58 +01:00
Thomas Basler
4053e31a5e Prevent config corruption by checking whether memory allocation was successfull. 2024-01-04 13:42:39 +01:00
Thomas Basler
c1fc907ecb Fix: Access Point not working after firmware update
Fixes #1613
2024-01-03 22:04:50 +01:00
Thomas Basler
cbbe053cd5 Fix: Optimize network connection handling
This should provide a more reliable connection to several AP types. See #576
2024-01-03 01:17:39 +01:00
helgeerbe
cbf7680836 merge v23.12.31 2024-01-01 14:50:51 +01:00
Bernhard Kirchen
8f5c4878c5
Fix: switch context when processing DPL MQTT requests (#572)
MQTT message callbacks are executed in the MQTT thread context. when
processing topics that control the DPL, we must avoid executing methods
that are not thread-safe. this change binds the methods to be called to
the respective parameters and executes them in the TaskScheduler
context, such that they no longer need to be thread-safe.
2023-12-31 14:49:39 +01:00
Bernhard Kirchen
463226082f clean up Huawei MQTT handler
* bind the callback to a topic (enum value) such that there is no need
  to tokenize the full topic (string) to find out what value is being
  processed. tokenizing is expensive.
* get rid of using the config in the callback, which improves
  thread-safety since the MQTT callback is running in the MQTT thread.
* prefer C++ method stof to convert MQTT value to a float, which saves
  us from using new and delete for a buffer in particular.
* prefer switch statements over if-else-trees.
* split long lines.
* get rid of topic #defines.
* fix indention.
2023-12-30 18:41:57 +01:00
Bernhard Kirchen
fe2f82e303 Fix: switch context when handling AC charger MQTT messages
MQTT message callbacks are executed in the MQTT thread context. when
processing topics that control the huawei AC charger, we must avoid
executing methods that are not thread-safe. this change bound the
methods to be called to the respective parameters and executes them
in the TaskScheduler context, such that they no longer need to be
thread-safe.
2023-12-30 18:08:06 +01:00
Bernhard Kirchen
7928f2f8cf
Feature: JK BMS: add more values to live view (#552)
there are more interesting values available to display in the live view.
however, adding them made the list of values very long. this can be
mitigated by using a new column/card, which uses the available screen
space nicely on bigger screens.
2023-12-30 16:46:44 +01:00
Bernhard Kirchen
c7098b6c42
Fix: thread-safety and dynamic memory for MessageOutput (#567)
this commit re-introduces the changes from #418, which were effectively
reverted with d49481097 (merge commit introducing TaskScheduler).

these adjustments are important to guarantee unmangled log messages and
more importantly, to guarantee that all messages from a particular
component are printed to the web console, which most people use to copy
messages from when reporting issues.

* use dynamic memory to allow handling of arbitrary message lenghts.
* keep a message buffer for every task so no task ever mangles the
  message of another task.
* every complete line is written to the serial console and moved to
  a line buffer for sending them through the websocket.
* the websocket is always fed complete lines.
* make sure to feed only as many lines as possible to the websocket
  handler, so that no lines are dropped.
* lock all MessageOutput state against concurrent access.
* respect HardwareSerial buffer size: the MessageOutput class buffers
  whole lines of output printed by any task in order to avoid mangling
  of text. that means we hand over full lines to the HardwareSerial
  instance, which might be too much in one call to write(buffer, size).
  we now check the return value of write(buffer, size) and call the
  function again with the part of the message that could not yet be
  written by HardwareSerial.
2023-12-30 16:45:56 +01:00
Thomas Basler
aa8068370e Feature: Add support for SSD1309 2.4" Display 2023-12-30 12:42:42 +01:00
helgeerbe
f89f9da67a Merge remote-tracking branch 'tbnobody/OpenDTU/master' into development 2023-12-29 10:54:41 +01:00
Bernhard Kirchen
d769cdd30a
Fix: move battery's lock_guard to updateSettings() (#566)
the updateSettings method is called from the web server's context and
therefore accesses _upProvider in a different context than the
TaskScheduler. the lock_guard needs to protect _upProvider.
2023-12-29 10:48:25 +01:00
Thomas Basler
34983471de Simplfy diagram drawing code 2023-12-28 00:10:40 +01:00
helgeerbe
6ab706c87d clean up merge conflicts 2023-12-27 13:16:37 +01:00
helgeerbe
367e0f9b6e Merge remote-tracking branch 'tbnobody/OpenDTU/master' into development 2023-12-27 13:11:25 +01:00
helgeerbe
d494810975
merge V23.12.16 (#556)
* Optimize Sun data calculation

* Remove not required enum

* Split config struct into different sub structs

* Feature: Allow configuration of LWT QoS

* Made resetreason methods static

* Feature: Implement offset cache for "YieldDay"

Thanks to @broth-itk for the idea!
Fix: #1258 #1397

* Add Esp32-Stick-PoE-A

* remove broken LilyGO_T_ETH_POE config, use device profile instead

* Feature: High resolution Icon and PWA (Progressive Web App) functionality

Fix: #1289

* webapp: Update dependencies

* Initialize TaskScheduler

* Migrate SunPosition to TaskScheduler

* Migrate Datastore to TaskScheduler

* Migrate MqttHandleInverterTotal to TaskSchedule

* Migrate MqttHandleHass to TaskScheduler

* Migrate MqttHandleDtu to TaskScheduler

* Migrate MqttHandleInverter to TaskScheduler

* Migrate LedSingle to TaskScheduler

* Migrate NetworkSettings to TaskScheduler

* Migrate InverterSettings to TaskScheduler

* Migrate MessageOutput to TaskScheduler

* Migrate Display_Graphic to TaskScheduler

* Migrate WebApi to TaskScheduler

* Split InverterSettings into multiple tasks

* Calculate SunPosition only every 5 seconds

* Split LedSingle into multiple tasks

* Upgrade espMqttClient from 1.4.5 to 1.5.0

* Doc: Correct amount of MPP-Tracker

* Added HMT-1600-4T and HMT-1800-4T to DevInfoParser

Fix #1524

* Adjusted inverter names for HMS-1600/1800/2000-4T

* Add channel count to description of detected inverter type (DevInfoParser)

* Adjust device web api endpoint for dynamic led count

* Feature: Added ability to change the brightness of the LEDs

Based on the idea of @moritzlerch with several modifications like pwmTable and structure

* webapp: Update dependencies

* Update olikraus/U8g2 from 2.35.7 to 2.35.8

* Remove not required onWebsocketEvent

* Remove code nesting

* Introduce several const statements

* Remove not required AsyncEventSource

* Doc: Added byte specification to each command

* Feature: Added basic Grid Profile parser which shows the used profile and version

Other values are still outstanding.

* Optimize AlarmLogParser to save memory

* Add libfrozen to project to create constexpr maps

* Feature: First version of GridProfile Parser which shows all values contained in the profile.

* webapp: Update dependencies

* Apply better variable names

* Remove not required casts

* Add additional compiler flags to prevent errors

* Add const statement to several variables

* Replace NULL by nullptr

* Update bblanchon/ArduinoJson from 6.21.3 to 6.21.4

* Add const keyword to method parameters

* Add const keyword to methods

* Use references instead of pointers whenver possible

* Adjust member variable names in MqttSettings

* Adjust member variable names in NetworkSettings

* webapp: Update timezone database to latest version

* webapp: Beautify and unify form footers

* Feature: Allow setting of an inverter limit of 0% and 0W

Thanks to @madmartin in #1270

* Feature: Allow links in device profiles

These links will be shown on the hardware settings page.

* Doc: Added hint regarding HMS-xxxx-xT-NA inverters

* Feature: Added DeviceProfile for CASmo-DTU

Based on #1565

* Upgrade actions/upload-artifact from v3 to v4

* Upgrade actions/download-artifact from v3 to v4

* webapp: add app.js.gz

* Gridprofileparser: Added latest known values

Thanks to @stefan123t and @noone2k

* webapp: Fix lint errors

* Feature: Add DTU to Home Assistant Auto Discovery

This is based on PR 1365 from @CFenner with several fixes and optimizations

* Fix: Remove debug output as it floods the console

* Fix: Gridprofileparser: Add additional error handling if profile is unknown

* webapp: add app.js.gz

* Fix: Offset cache for "YieldDay" did not work correctly

* webapp: update dependencies

* webapp: add app.js.gz

* Fix: yarn.lock was outdated

* Fix: yarn build error

* Fix: Reset Yield day correction in combination with Zero Yield Day on Midnight lead to wrong values.

* Fix: Allow negative values in GridProfileParser

* Correct variable name

* Fix #1579: Static IP in Ethernet mode did not work correctly

* Feature: Added diagram to display

This is based on the idea of @Henrik-Ingenieur and was discussed in #1504

* webapp: update dependencies

* webapp: add app.js.gz

---------

Co-authored-by: Thomas Basler <thomas@familie-basler.net>
Co-authored-by: Pierre Kancir <pierre.kancir.emn@gmail.com>
2023-12-27 11:49:57 +01:00
Thomas Basler
06b8fb65a1 Move the diagram in Y direction if screen saver mode is turned on 2023-12-24 15:08:10 +01:00
Thomas Basler
2259b1b525 Add const keyword to variable 2023-12-24 14:35:29 +01:00
Stefan Oberhumer
ac427523b5 Feature: Add screensavermode also to diagram drawing. 2023-12-24 12:24:10 +01:00
Stefan Oberhumer
733a566172 NFC: Use std::size() instead of define. 2023-12-24 12:13:57 +01:00
Stefan Oberhumer
c1f0b9ea6f http response header tag 'ETag' needs quotes.
As per RFC7232 ( https://www.rfc-editor.org/rfc/rfc7232#section-2.3 ) and
MDN docs ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag#directives )
the value of the ETag in the http header response has to be between quotes.
2023-12-23 19:00:44 +01:00
Thomas Basler
0ddc7fd28d Feature: Added diagram to display
This is based on the idea of @Henrik-Ingenieur and was discussed in #1504
2023-12-19 17:26:24 +01:00
Thomas Basler
3b923885de Fix #1579: Static IP in Ethernet mode did not work correctly 2023-12-19 11:30:33 +01:00
Thomas Basler
84248ec9b6 Feature: Add DTU to Home Assistant Auto Discovery
This is based on PR 1365 from @CFenner with several fixes and optimizations
2023-12-17 14:40:04 +01:00
Bernhard Kirchen
6e78c5bd1c
Feature: JK BMS: export (more) data to live view and MQTT (#549)
* add more values to web app live view. this should add all interesting
  values for the web app live view. those include important values and
  values that change frequently.

* add more interesting JK BMS dummy messages: one has 0% SoC and an
  alarm (discharge undervoltage) set. the other has the undertemperature
  alarm set.

* add alarms and warnings to live view

* publish alarm and status bits through MQTT individually

* publish cell voltages to MQTT broker

* remove trailing spaces in BatteryStats class
2023-12-15 10:59:07 +01:00
Thomas Basler
b4ead9d6ec Feature: Allow setting of an inverter limit of 0% and 0W
Thanks to @madmartin in #1270
2023-12-13 22:15:49 +01:00
Thomas Basler
4f7597eaa1 Adjust member variable names in NetworkSettings 2023-12-12 23:55:03 +01:00
Thomas Basler
66530638de Adjust member variable names in MqttSettings 2023-12-12 23:54:58 +01:00
Thomas Basler
e95afbf2cc Use references instead of pointers whenver possible 2023-12-12 13:26:07 +01:00
Thomas Basler
8fb43fedf8 Add const keyword to methods 2023-12-12 01:30:40 +01:00
Thomas Basler
d6494fa791 Add const keyword to method parameters 2023-12-12 00:21:14 +01:00
Thomas Basler
5e45006831 Replace NULL by nullptr 2023-12-11 20:23:45 +01:00
Thomas Basler
47e905bcfc Add const statement to several variables 2023-12-11 19:54:57 +01:00
Thomas Basler
efe0b6b383 Remove not required casts 2023-12-11 18:44:02 +01:00
Thomas Basler
06651f365a Feature: First version of GridProfile Parser which shows all values contained in the profile. 2023-12-11 14:58:17 +01:00
Thomas Basler
00bc631e87 Feature: Added basic Grid Profile parser which shows the used profile and version
Other values are still outstanding.
2023-12-09 11:12:37 +01:00
Thomas Basler
b937532505 Remove not required AsyncEventSource 2023-12-07 15:21:24 +01:00
Thomas Basler
8b5d406a4f Introduce several const statements 2023-12-07 15:19:04 +01:00