Error when running 'brewblox-ctl up'

Troubleshooting
#1

Hi! Installed brewblox for the first time in a new rbpi following the documentation steps. There were some hiccups when updating the packages, but after brewblox was apparently installed without problems.

But when after I tried to run the spark simulation I got some errors after running brewblox-ctl up. This is the first and then some other exceptions follow:

Traceback (most recent call last):
File “/home/pi/brewblox/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 699, in urlopen
httplib_response = self._make_request(
File “/home/pi/brewblox/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File “/usr/lib/python3.9/http/client.py”, line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File “/usr/lib/python3.9/http/client.py”, line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File “/usr/lib/python3.9/http/client.py”, line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File “/usr/lib/python3.9/http/client.py”, line 1010, in _send_output
self.send(msg)
File “/usr/lib/python3.9/http/client.py”, line 950, in send
self.connect()
File “/home/pi/brewblox/.venv/lib/python3.9/site-packages/docker/transport/unixconn.py”, line 30, in connect
sock.connect(self.unix_socket)
ConnectionRefusedError: [Errno 111] Connection refused

Already re-formated the SD card and tried again, but encountered the same problem. A pointer in the right direction would be very appreciated…

Thanks!

#2

What were the hiccups during installation? It’s complaining it can’t talk to the Docker socket. What is the output of sudo systemctl status docker?

#3

Thanks for the incredible fast response :slight_smile: .

This is the output of ‘sudo systemctl status docker’:

● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2021-11-28 22:17:12 WET; 4min 38s ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 674 (dockerd)
Tasks: 10
CPU: 3.841s
CGroup: /system.slice/docker.service
└─674 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Nov 28 22:17:09 rbpi2 dockerd[674]: time=“2021-11-28T22:17:09.420236557Z” level=warning msg=“Unable to find memory controller”
Nov 28 22:17:09 rbpi2 dockerd[674]: time=“2021-11-28T22:17:09.423107651Z” level=info msg=“Loading containers: start.”
Nov 28 22:17:10 rbpi2 dockerd[674]: time=“2021-11-28T22:17:10.862211609Z” level=info msg="Default bridge (docker0) is assigned with an IP address 172.17>
Nov 28 22:17:11 rbpi2 dockerd[674]: time=“2021-11-28T22:17:11.379712129Z” level=info msg=“Loading containers: done.”
Nov 28 22:17:12 rbpi2 dockerd[674]: time=“2021-11-28T22:17:12.517349316Z” level=info msg=“Docker daemon” commit=847da18 graphdriver(s)=overlay2 version=>
Nov 28 22:17:12 rbpi2 dockerd[674]: time=“2021-11-28T22:17:12.519451816Z” level=info msg=“Daemon has completed initialization”
Nov 28 22:17:12 rbpi2 systemd[1]: Started Docker Application Container Engine.
Nov 28 22:17:12 rbpi2 dockerd[674]: time=“2021-11-28T22:17:12.758036295Z” level=info msg=“API listen on /run/docker.sock”
Nov 28 22:21:06 rbpi2 dockerd[674]: time=“2021-11-28T22:21:06.005957120Z” level=error msg="Handler for GET /v1.41/images/brewblox/brewblox-devcon-spark:>
Nov 28 22:21:31 rbpi2 dockerd[674]: time=“2021-11-28T22:21:31.788242825Z” level=error msg="Handler for GET /v1.41/images/brewblox/brewblox-devcon-spark:>

The hiccups were after ‘sudo apt update && sudo apt upgrade -y’, some of the updating steps didn’t finish. But after I tried ‘apt-get update’ and ‘apt-get install --fix-missing’ and I thought I had solved it, but maybe not… If I now run ‘sudo apt update && sudo apt upgrade -y’ I get:

Hit:1 Index of /raspbian bullseye InRelease
Hit:2 Index of /debian bullseye InRelease
Hit:3 Index of linux/raspbian/ bullseye InRelease
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
All packages are up to date.
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
Calculating upgrade… Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Also now, after rebooting, if I try to activate the simulation, I get:

ERROR: layer does not exist
Command ‘docker-compose up -d’ returned non-zero exit status 1.

Hope this was useful…

#4

The output from systemctl looks normal.

To check docker itself:

docker run --rm hello-world

This should print a welcome message, and then exit.

To recheck and restart your services, run this in your brewblox dir:

source .venv/bin/activate
docker-compose down
docker-compose pull
docker-compose up -d
#5

Everything runs as expected until ‘docker-compose up -d’, then I get:

Creating network “brewblox_default” with the default driver

ERROR: layer does not exist

#6

Docker run results in error: layer does not exist - Stack Overflow suggests that a hard reset of docker data solved this error.

This will remove all stored images, and cause them to be re-downloaded when you run the up command.

brewblox-ctl down
sudo service docker stop
sudo rm -rf /var/lib/docker
sudo service docker start
brewblox-ctl up
#7

Thanks, it worked! I had actually found the SO post, but was clueless about what to do. Again, thanks a lot!

#8

Happy to hear it works! Interpreting search results is not trivial either, so don’t worry about that - the URL is mostly for reference.

#9

Hello,
I didn´t find “new topic” so I write here instead.
I have raspberry 4 and I am trying to set it up.
On step 5 of the setup, I have the following error when starting the brewblox :

brewblox-ctl up
ERROR: readlink /var/lib/docker/overlay2: invalid argument
Command 'docker-compose up -d ’ returned non-zero exit status 1.

Any advice or way to solve it ?
thanks a lot

#10

This can happen if a docker image was corrupted during download. To fix this, run

docker system prune --all
brewblox-ctl up
#11

downloads where completed but
now there is this error:
Status: Downloaded newer image for brewblox/brewblox-devcon-spark:edge
Pulling redis (redis:6.0)…
ERROR: Head “https://registry-1.docker.io/v2/library/redis/manifests/6.0”: Get “https://auth.docker.io/token?scope=repository%3Alibrary%2Fredis%3Apull&service=registry.docker.io”: context deadline exceeded
Command 'docker-compose up -d ’ returned non-zero exit status 1.
pi@raspberrypi:~/brewblox $

#12

Does it happen again if you retry? A timeout error is usually due to connection issues or server hiccups.

#13

hey thanks for your super fast reply.
and sorry iam very new to it.
great work you are doing thankx for that.
ill try again
in this state raspberrypi and spark 4 doesnt need to be connected with usb cabel ?

#14

The spark uses either USB or wifi for communication. If you have set up wifi, you can disconnect the Spark from the Pi.

#15

Small correction: on the Spark 4, we don’t use USB for communication. If you want wired instead of wireless, it has Ethernet.

#16

Not sure if the source of your errors is the same as mine, but I kept having similar problems until I changed SD card. Never understood if it was corrupted or if it just didn’t worked, apparently some cards are more compatible with the rbpi than others (RPi SD cards - eLinux.org).

So if your problems persist after trying Bob’s suggestions, maybe you should try a different card.

Hope this helps and that I’m not just creating more confusion…

#17

This can be the cause, yes. If it happens once, it’s probably a connection or server issue. If it keeps happening, it is more likely to be a card issue.

SD cards vary wildly in read/write speed and quality. A Raspberry Pi will use a card much more intensively than a digital camera would.