Validating controller handshake

1

Hello

I have been running my brewery with the Spark 3 for some time now. The other day, out of nowhere, I couldn’t connect to it anymore,

I have looked through all past threads for help, and have tried them all, but it seems to go back where i started: in the UI, it can find the Spark and connect to it, but it stalls when validating controller handshake.

The other day i was succesful in setting up WIFI through the Particle app, but now it gives me an error message - after several attempts.

I have done a full re-install on brewblox on my PI, I have made sure the power supply is 2,4A (in accordance with what I have read here). Is the Spark dead?

Here’s the log from the Spark:

+++++++++++++++++++++++++++++++++++++++++++++++++ Service: sjaiber-spark +++++++++++++++++++++++++++++++++++++++++++++++++

Attaching to brewblox_sjaiber-spark_1
sjaiber-spark_1 | 2023-01-25T08:35:53.505990637Z 2023/01/25 09:35:53 INFO main firmware.ini: {‘firmware_version’: ‘10bb1f6d’, ‘firmware_date’: ‘2023-01-06’, ‘firmware_sha’: ‘10bb1f6d0c63129e333ba4d881cfc649e9c1cae4’, ‘proto_version’: ‘9601ba60’, ‘proto_date’: ‘2023-01-06’, ‘system_version’: ‘3.2.0’}
sjaiber-spark_1 | 2023-01-25T08:35:53.537796090Z 2023/01/25 09:35:53 INFO brewblox_service.service Service name: sjaiber-spark
sjaiber-spark_1 | 2023-01-25T08:35:53.537975415Z 2023/01/25 09:35:53 INFO brewblox_service.service Service info: @ Mon Jan 9 15:21:17 UTC 2023
sjaiber-spark_1 | 2023-01-25T08:35:53.538014635Z 2023/01/25 09:35:53 INFO brewblox_service.service Service config: {‘host’: ‘0.0.0.0’, ‘port’: 5000, ‘name’: ‘sjaiber-spark’, ‘debug’: False, ‘mqtt_protocol’: ‘mqtt’, ‘mqtt_host’: ‘eventbus’, ‘mqtt_port’: None, ‘mqtt_path’: ‘/eventbus’, ‘history_topic’: ‘brewcast/history’, ‘state_topic’: ‘brewcast/state’, ‘simulation’: False, ‘device_host’: None, ‘device_port’: 8332, ‘device_serial’: None, ‘device_id’: ‘1E003A001147383531363134’, ‘discovery’: ‘all’, ‘command_timeout’: 20, ‘broadcast_interval’: 5, ‘volatile’: False, ‘datastore_topic’: ‘brewcast/datastore’, ‘skip_version_check’: False, ‘backup_interval’: 3600, ‘backup_retry_interval’: 300}
sjaiber-spark_1 | 2023-01-25T08:35:53.538056666Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt Starting <EventHandler for mqtt://eventbus:1883>
sjaiber-spark_1 | 2023-01-25T08:35:53.538088542Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt listen(brewcast/datastore/brewblox-global)
sjaiber-spark_1 | 2023-01-25T08:35:53.538119480Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt subscribe(brewcast/datastore/brewblox-global)
sjaiber-spark_1 | 2023-01-25T08:35:53.538149168Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt listen(brewcast/spark/blocks/create)
sjaiber-spark_1 | 2023-01-25T08:35:53.538178699Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt listen(brewcast/spark/blocks/write)
sjaiber-spark_1 | 2023-01-25T08:35:53.538208023Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt listen(brewcast/spark/blocks/patch)
sjaiber-spark_1 | 2023-01-25T08:35:53.538237346Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt listen(brewcast/spark/blocks/delete)
sjaiber-spark_1 | 2023-01-25T08:35:53.538267190Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt subscribe(brewcast/spark/blocks/#)
sjaiber-spark_1 | 2023-01-25T08:35:53.573506906Z 2023/01/25 09:35:53 INFO brewblox_service.mqtt <EventHandler for mqtt://eventbus:1883> connected
sjaiber-spark_1 | 2023-01-25T08:35:53.611392436Z 2023/01/25 09:35:53 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:35:53.639188360Z 2023/01/25 09:35:53 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:35:53.640661555Z 2023/01/25 09:35:53 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:35:53.666413755Z 2023/01/25 09:35:53 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:35:54.693038900Z 2023/01/25 09:35:54 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:35:54.733446025Z 2023/01/25 09:35:54 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:35:54.782881076Z 2023/01/25 09:35:54 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:35:54.851247772Z 2023/01/25 09:35:54 INFO …evcon_spark.synchronization System uptime: 6m29s
sjaiber-spark_1 | 2023-01-25T08:35:54.851941218Z 2023/01/25 09:35:54 INFO …evcon_spark.synchronization System updates per second: 20.674
sjaiber-spark_1 | 2023-01-25T08:35:56.134231152Z 2023/01/25 09:35:56 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:35:56.135163195Z 2023/01/25 09:35:56 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:35:56.137141293Z 2023/01/25 09:35:56 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:35:56.144429770Z 2023/01/25 09:35:56 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:35:56.146318805Z 2023/01/25 09:35:56 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:35:56.148201016Z 2023/01/25 09:35:56 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:35:56.149245404Z 2023/01/25 09:35:56 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:35:56.178611608Z 2023/01/25 09:35:56 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:35:56.204825949Z 2023/01/25 09:35:56 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:35:58.153113404Z 2023/01/25 09:35:58 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:35:58.187805039Z 2023/01/25 09:35:58 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:35:59.216938705Z 2023/01/25 09:35:59 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:35:59.253966933Z 2023/01/25 09:35:59 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:35:59.296346999Z 2023/01/25 09:35:59 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:35:59.316706527Z 2023/01/25 09:35:59 INFO …evcon_spark.synchronization System uptime: 6m33s
sjaiber-spark_1 | 2023-01-25T08:35:59.317059448Z 2023/01/25 09:35:59 INFO …evcon_spark.synchronization System updates per second: 14.527
sjaiber-spark_1 | 2023-01-25T08:35:59.819928347Z 2023/01/25 09:35:59 INFO …ox_devcon_spark.block_store Saved 6 block(s)
sjaiber-spark_1 | 2023-01-25T08:36:00.607000260Z 2023/01/25 09:36:00 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:00.607523704Z 2023/01/25 09:36:00 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:00.609612428Z 2023/01/25 09:36:00 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:00.617070439Z 2023/01/25 09:36:00 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:00.621225283Z 2023/01/25 09:36:00 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:00.621438255Z 2023/01/25 09:36:00 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:00.621525495Z 2023/01/25 09:36:00 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:00.648553285Z 2023/01/25 09:36:00 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:00.676234311Z 2023/01/25 09:36:00 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:02.625738968Z 2023/01/25 09:36:02 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:02.661689839Z 2023/01/25 09:36:02 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:03.696560660Z 2023/01/25 09:36:03 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:03.701003998Z 2023/01/25 09:36:03 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:03.743897768Z 2023/01/25 09:36:03 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:03.812283683Z 2023/01/25 09:36:03 INFO …evcon_spark.synchronization System uptime: 6m38s
sjaiber-spark_1 | 2023-01-25T08:36:03.812603167Z 2023/01/25 09:36:03 INFO …evcon_spark.synchronization System updates per second: 14.755
sjaiber-spark_1 | 2023-01-25T08:36:05.102950390Z 2023/01/25 09:36:05 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:05.113422814Z 2023/01/25 09:36:05 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:05.113755266Z 2023/01/25 09:36:05 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:05.113830840Z 2023/01/25 09:36:05 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:05.116060816Z 2023/01/25 09:36:05 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:05.116237016Z 2023/01/25 09:36:05 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:05.116302329Z 2023/01/25 09:36:05 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:05.144846179Z 2023/01/25 09:36:05 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:05.176801843Z 2023/01/25 09:36:05 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:07.120274028Z 2023/01/25 09:36:07 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:07.155039206Z 2023/01/25 09:36:07 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:08.189118247Z 2023/01/25 09:36:08 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:08.201442986Z 2023/01/25 09:36:08 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:08.240033109Z 2023/01/25 09:36:08 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:08.266928240Z 2023/01/25 09:36:08 INFO …evcon_spark.synchronization System uptime: 6m42s
sjaiber-spark_1 | 2023-01-25T08:36:08.267377620Z 2023/01/25 09:36:08 INFO …evcon_spark.synchronization System updates per second: 14.902
sjaiber-spark_1 | 2023-01-25T08:36:08.782936940Z 2023/01/25 09:36:08 INFO …ox_devcon_spark.block_store Saved 6 block(s)
sjaiber-spark_1 | 2023-01-25T08:36:09.553474217Z 2023/01/25 09:36:09 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:09.553917087Z 2023/01/25 09:36:09 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:09.555590285Z 2023/01/25 09:36:09 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:09.562750376Z 2023/01/25 09:36:09 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:09.563724346Z 2023/01/25 09:36:09 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:09.565200667Z 2023/01/25 09:36:09 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:09.566804021Z 2023/01/25 09:36:09 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:09.593727642Z 2023/01/25 09:36:09 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:09.628288547Z 2023/01/25 09:36:09 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:11.570637853Z 2023/01/25 09:36:11 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:11.605882933Z 2023/01/25 09:36:11 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:12.638516331Z 2023/01/25 09:36:12 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:12.648698595Z 2023/01/25 09:36:12 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:12.690281672Z 2023/01/25 09:36:12 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:12.711899914Z 2023/01/25 09:36:12 INFO …evcon_spark.synchronization System uptime: 6m47s
sjaiber-spark_1 | 2023-01-25T08:36:12.712361638Z 2023/01/25 09:36:12 INFO …evcon_spark.synchronization System updates per second: 14.828
sjaiber-spark_1 | 2023-01-25T08:36:14.005460406Z 2023/01/25 09:36:14 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:14.014458124Z 2023/01/25 09:36:14 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:14.014643804Z 2023/01/25 09:36:14 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:14.015387876Z 2023/01/25 09:36:14 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:14.019580585Z 2023/01/25 09:36:14 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:14.019770848Z 2023/01/25 09:36:14 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:14.019839963Z 2023/01/25 09:36:14 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:14.042659730Z 2023/01/25 09:36:14 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:14.070315444Z 2023/01/25 09:36:14 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:16.023801297Z 2023/01/25 09:36:16 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:16.058489652Z 2023/01/25 09:36:16 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:17.089243859Z 2023/01/25 09:36:17 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:17.097618444Z 2023/01/25 09:36:17 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:17.139704860Z 2023/01/25 09:36:17 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:17.164585226Z 2023/01/25 09:36:17 INFO …evcon_spark.synchronization System uptime: 6m51s
sjaiber-spark_1 | 2023-01-25T08:36:17.165068253Z 2023/01/25 09:36:17 INFO …evcon_spark.synchronization System updates per second: 14.824
sjaiber-spark_1 | 2023-01-25T08:36:17.715174728Z 2023/01/25 09:36:17 INFO …ox_devcon_spark.block_store Saved 6 block(s)
sjaiber-spark_1 | 2023-01-25T08:36:18.451199330Z 2023/01/25 09:36:18 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:18.451528657Z 2023/01/25 09:36:18 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:18.452911382Z 2023/01/25 09:36:18 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:18.459809334Z 2023/01/25 09:36:18 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:18.460875389Z 2023/01/25 09:36:18 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:18.462315251Z 2023/01/25 09:36:18 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:18.463327868Z 2023/01/25 09:36:18 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:18.492871262Z 2023/01/25 09:36:18 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:18.519355815Z 2023/01/25 09:36:18 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:20.468688074Z 2023/01/25 09:36:20 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:20.504125396Z 2023/01/25 09:36:20 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:21.538130582Z 2023/01/25 09:36:21 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:21.571700381Z 2023/01/25 09:36:21 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:21.616634958Z 2023/01/25 09:36:21 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:21.686249691Z 2023/01/25 09:36:21 INFO …evcon_spark.synchronization System uptime: 6m56s
sjaiber-spark_1 | 2023-01-25T08:36:21.686604852Z 2023/01/25 09:36:21 INFO …evcon_spark.synchronization System updates per second: 14.844
sjaiber-spark_1 | 2023-01-25T08:36:22.974896164Z 2023/01/25 09:36:22 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:22.980256544Z 2023/01/25 09:36:22 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:22.980464151Z 2023/01/25 09:36:22 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:22.984573786Z 2023/01/25 09:36:22 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:22.985251399Z 2023/01/25 09:36:22 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:22.986542509Z 2023/01/25 09:36:22 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:22.987553146Z 2023/01/25 09:36:22 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:23.013342274Z 2023/01/25 09:36:23 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:23.042288473Z 2023/01/25 09:36:23 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:24.991548387Z 2023/01/25 09:36:24 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:25.025566160Z 2023/01/25 09:36:25 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:26.054710660Z 2023/01/25 09:36:26 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:26.069519232Z 2023/01/25 09:36:26 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:26.111202779Z 2023/01/25 09:36:26 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:26.132711488Z 2023/01/25 09:36:26 INFO …evcon_spark.synchronization System uptime: 7m
sjaiber-spark_1 | 2023-01-25T08:36:26.133291547Z 2023/01/25 09:36:26 INFO …evcon_spark.synchronization System updates per second: 14.809
sjaiber-spark_1 | 2023-01-25T08:36:26.654575522Z 2023/01/25 09:36:26 INFO …ox_devcon_spark.block_store Saved 6 block(s)
sjaiber-spark_1 | 2023-01-25T08:36:27.426636569Z 2023/01/25 09:36:27 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:27.427502673Z 2023/01/25 09:36:27 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:27.429245977Z 2023/01/25 09:36:27 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:27.438449635Z 2023/01/25 09:36:27 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:27.438690835Z 2023/01/25 09:36:27 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:27.439097403Z 2023/01/25 09:36:27 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:27.440788258Z 2023/01/25 09:36:27 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:27.464439390Z 2023/01/25 09:36:27 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:27.493901428Z 2023/01/25 09:36:27 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:29.444591204Z 2023/01/25 09:36:29 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:29.480587909Z 2023/01/25 09:36:29 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, ‘USB VID:PID=2B04:C008 SER=1e003a001147383531363134 LOCATION=1-1.4:1.0’]
sjaiber-spark_1 | 2023-01-25T08:36:30.514851379Z 2023/01/25 09:36:30 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> connected
sjaiber-spark_1 | 2023-01-25T08:36:30.520843121Z 2023/01/25 09:36:30 INFO …lox_devcon_spark.connection HandshakeMessage(name=‘BREWBLOX’, firmware_version=‘10bb1f6d’, proto_version=‘9601ba60’, firmware_date=‘2023-01-06’, proto_date=‘2023-01-06’, system_version=‘3.2.0’, platform=‘p1’, reset_reason_hex=‘78’, reset_data_hex=‘00’, device_id=‘1E003A001147383531363134’, reset_reason=‘DFU_MODE’, reset_data=‘NOT_SPECIFIED’)
sjaiber-spark_1 | 2023-01-25T08:36:30.560717531Z 2023/01/25 09:36:30 INFO …ox_devcon_spark.block_store Read 6 blocks
sjaiber-spark_1 | 2023-01-25T08:36:30.585933735Z 2023/01/25 09:36:30 INFO …evcon_spark.synchronization System uptime: 7m5s
sjaiber-spark_1 | 2023-01-25T08:36:30.586576138Z 2023/01/25 09:36:30 INFO …evcon_spark.synchronization System updates per second: 14.824
sjaiber-spark_1 | 2023-01-25T08:36:31.875511000Z 2023/01/25 09:36:31 ERROR …evcon_spark.synchronization Sync subroutine failed: sync controller settings - CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:31.876517523Z 2023/01/25 09:36:31 ERROR …evcon_spark.synchronization Failed to sync: CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:31.880484969Z 2023/01/25 09:36:31 ERROR brewblox_service.repeater error during run(): CommandException(BLOCK_WRITE, INSUFFICIENT_STORAGE)
sjaiber-spark_1 | 2023-01-25T08:36:31.885513678Z 2023/01/25 09:36:31 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> closed stream writer
sjaiber-spark_1 | 2023-01-25T08:36:31.886938071Z 2023/01/25 09:36:31 INFO …lox_devcon_spark.connection <SparkConnection for /dev/ttyACM0> terminated subprocess
sjaiber-spark_1 | 2023-01-25T08:36:31.889059869Z 2023/01/25 09:36:31 ERROR brewblox_service.repeater <SparkConnection for /dev/ttyACM0> error during run(): ConnectionError(EOF received)
sjaiber-spark_1 | 2023-01-25T08:36:31.891446045Z 2023/01/25 09:36:31 INFO …lox_devcon_spark.connection Retrying connection…
sjaiber-spark_1 | 2023-01-25T08:36:31.916615686Z 2023/01/25 09:36:31 INFO …_devcon_spark.service_store read {‘namespace’: ‘spark-service’, ‘id’: ‘sjaiber-spark-service-db’, ‘data’: {‘autoconnecting’: True, ‘retry_interval’: 2}}
sjaiber-spark_1 | 2023-01-25T08:36:31.949907356Z 2023/01/25 09:36:31 INFO …evcon_spark.synchronization Service temperature unit set to degC
sjaiber-spark_1 | 2023-01-25T08:36:33.894662109Z 2023/01/25 09:36:33 INFO …_devcon_spark.connect_funcs Discovering devices… (all)
sjaiber-spark_1 | 2023-01-25T08:36:33.928596496Z 2023/01/25 09:36:33 INFO …_devcon_spark.connect_funcs Discovered [‘/dev/ttyACM0’, ‘P1 - P1 Serial’, 'USB

Hope you can help

Regards

-Per

2

The error that you are are getting is because the persistent storage for settings on the spark is full and it cannot relocate a block store a setting.
Did you have a large amount of blocks on the spark? If not it might be corruption of the data that requires a wipe.

3

Hi Elco

thanks for the reply. I would say i do have a large amount of blocks. But as i cannot acces the spark, I can’t delete them. Is there another way of doing this?

thanks

4

We can send manual commands to the Spark to clear up, and avoid a full wipe.
Do you have a recent block backup? If you do, please upload it.

If you don’t, please look up the IP of your Spark, and then run the following commands. Copy-paste the output here (no screenshots).

sudo apt install -y netcat
brewblox-ctl down
nc {IP ADDRESS OF YOUR SPARK} 8332

This will open a direct connection to your Spark. Press enter, and it should print a handshake.

Copy the following characters, and press Enter:

COgHEBU=

After it responds, you can press Ctrl-C to close the connection.

Example output:

(.venv) pi@manyberry:~/brewblox $ nc 192.168.2.2 8332

<!BREWBLOX,0c76feb1,98742fbf,2023-01-11,2023-01-10,"",esp32,00,00,C4DD5766BB18><!BREWBLOX,0c76feb1,98742fbf,2023-01-11,2023-01-10,"",esp32,00,00,C4DD5766BB18>
COgHEBU=
GgsIZBDFAiIERUFFPQ==,GgsIZRDFAiIERUFJPQ==,GgsIZhDFAiIERUFNPQ==,GhcIZxCuAiIQSVNnSWNTb05BQUJrS0dZPQ==,GhcIaBCuAiIQSVNqNzl2OElBQUJnS0dZPQ==,GhcIaRCuAiIQSVNqM1hDb05BQUFtS0dZPQ==,GhcIahC7AiIQQ1RvRUlTRUFBQUNKUUdZPQ==,GhcIaxCuAiIQSVNnWktGTUdBQUNVS0dRPQ==,Gg8IbBC+AiIIQ0dvUUFRPT0=,GiMIBxC6AiIcQ2hNSUFSSURRV25oR2doVFpXNXpiM0lnTVZCbg==,GjMIAhCAAiIsY2hwRFJWUXRNVU5GVTFRc1RUTXVOUzR3TEUweE1DNDFMakF2TTRBQi93RT0=,GhcIbRCvAiIQT0FGQWdJQUtTQUZRZ01BQw==,COgH

This is an encoded list of all stored blocks on your controller. This can be used to generate commands to remove one or more of them and make your Spark operable again.

For the long run, I’ll have a look at implementing a more automatic solution to this edge case.

5

Hi Bob

Thanks for your thorough reply.

I did make some backups:
brewblox_backup_20230123_1753.zip (5.1 KB)

hope that works

-Per

6

It looks like your Spark still contains a variety of blocks from previous systems (block name starts with New|).

Follow the above instructions to install netcat and open a connection, and then send:

COkHEA4aAwiZAQ==

Please copy the response, and post it here if the problem persists - it’s not guaranteed that the block still exists on the controller.

This will remove a block listed as New|SetpointProfile-3 in your backup, and free up enough space so your service can connect normally.
Afterwards, I suggest removing unused blocks in the Spark service page to free up space.

7

Hi Bob

I’ve now tried what you described, and I cannot seem to get the response you describe. Netcat is installed succesfully, but when i try to establish the connection, i get nothing.

Here’s what happened:

sjaiber@raspberrypi:~/brewblox $ sudo apt install -y netcat
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
netcat is already the newest version (1.10-46).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
sjaiber@raspberrypi:~/brewblox $ brewblox-ctl down
Removing network brewblox_default
WARNING: Network brewblox_default not found.
sjaiber@raspberrypi:~/brewblox $ brewblox-ctl down
Removing network brewblox_default
WARNING: Network brewblox_default not found.
sjaiber@raspberrypi:~/brewblox $ nc 192.168.0.113 8382
sjaiber@raspberrypi:~/brewblox $

Then, nothing…

-Per

8

The port in the nc command should be 8332, not 8382

9

Hi Bob

Ah yes, indeed. I did it, and here’s what came out:

sjaiber@raspberrypi:~/brewblox $ nc 192.168.0.113 8332

COkHEA4aAwiZAQ==

COkH

-Per

10

Hi again

That seemed to do it - thanks for your time and help. I’ll get the Spark cleaned up.

Good evening to you.

-Per

2 Likes