Legacy using Docker and legacy install issues

citybaker · May 23, 2019, 8:54am

Hey there,
Super noob here. I’m currently setting up RPi and Arduino to run legacy BrewPi.
I’ve been going through docs and this community and picking up pieces of the puzzle to put everything together. It’s a bit tricky because I’m not a programmer and not really used to the Unix OS either for command line shenanigans. I have been googling to learn what docker is and what containers are but everything is written for smart tech people and nothing is simplified in a way that a non programmer does I guess. But I can sort of understand what’s going on.

This is where I’m at - I used the official Docs to deploy the BrewPi container and then used code by Elco from another thread, which involved checking out the legacy version using Docker and bash, with a shallowclone flag. Trust me this sounds like I know what I’m doing but I don’t. Therefore -

I’m just unsure if I did everything correctly. Is there any way to be sure that I have installed the legacy version?
There was mention of a way to create a new image (or container?) with legacy version with the above docker code. How do I do that? What’s the benefit of this?
Is there any way to do any of this via the Docker GUI?

Thanks for all your help in advance. I’m learning a lot. I would buy Spark but I don’t live in the US and also at an early stage in the brewing rabithole so if this works out well then I’ll definitely be more inclined towards making the leap and getting a Spark when I visit the states next.

Bob_Steers · May 23, 2019, 10:14am

Disclaimer 1: I barely know anything about the (old) Brewpi software, let alone the legacy version of that - I was hired to build the new toys.

Disclaimer 2: We don’t officially support the legacy version anymore, and installing it will require figuring out a lot of software stuff. In the new software we made the installation process much simpler. You’re not directly using Docker, but get a simple menu to install, flash, start, and update your system.

That said, I can at least try and clarify all the magic words.

The BrewPi system uses two computers: a Raspberry Pi, and a controller (Spark or Arduino).
The controller is slow, but has hardware ports for sensors / actuators. This is why we use both: the controller controls the hardware, and the Pi runs the heavier software (web server).

You install new software on the controller by flashing a binary file. The controller will only run this binary code, nothing else. The software on the Pi consists of multiple files, and requires a separate program to run it. This is where Docker comes in.

A Docker image is a pre-packaged environment for software: it includes all files, and the programs to run them. Instead of a dozen files + a program you have one box with a “start” button.

The image itself can’t be started. It’s a blueprint for creating Containers. You can use a single image to start multiple independent Containers. Changing anything in a Container will not change the Image. If you stop and remove Containers, they’re gone: no leftover files, programs, or settings on your computer.

Could you please post links to the various guides / posts you are referring to? I can have a look, and point you to the relevant parts. There are some ways to check the exact version, but if you manage to get it running on an Arduino, you have the right one.

The benefit of using Docker is that you can experiment and debug without leftovers. It also makes sure you have the right version of all required programs.

I’m not very familiar with the various Docker GUIs, but usually they’re mostly for running containers from existing images, not building new images.

Not sure why you’d have to buy a Spark in the US: we’re based in the Netherlands, and ship internationally to pretty much anywhere except Russia.

citybaker · May 23, 2019, 11:56am

I’m sorry I assumed that brewPi was based in the US. My bad. Also, great to hear about international shipping as well.

Back on topic -
Thanks for the explanation on dockers, images and containers. That was super helpful. I think I understand those concepts a bit more now.

I used this guide to setup my RaspberryPi with Raspian, all the way up to deploying the brewPi container.

I used a hex uploader to upload this hex file to the Arduino. That went successfully too because the hex loader did seem to be working and it acknowledged the completion of the process.

Then, I launched a web browser to check if brewPi was working and it was.

Next I’m looking for the best way to downgrade to the legacy version, so this is what I did:

pi@bakepi:~ $ docker exec -it brewpi /bin/bash
root@1c88e2c00c7b:/home/brewpi# cd ~/brewpi-tools/
root@1c88e2c00c7b:~/brewpi-tools# sudo python updater.py --ask

Using interactive (advanced) update with user input 


######################################################
####                                              ####
####        Welcome to the BrewPi Updater!        ####
####                                              ####
######################################################

Checking whether the update script is up to date
/root/brewpi-tools is up-to-date.

It is not recommended to update during a brew!
If you are actively logging a brew we recommend canceling the the update with ctrl-c.


*** Updating BrewPi script repository ***

Stopping running instances of BrewPi
Quit message sent to BrewPi instance with pid 163!
You are on branch master

Available branches on the remote 'origin' for /home/brewpi:
[0] master
[1] Skip updating this repository
Enter the number of the branch you wish to update [master]:1


*** Updating BrewPi web interface repository ***
You are on branch master

Available branches on the remote 'origin' for /var/www/html:
[0] master
[1] Skip updating this repository
Enter the number of the branch you wish to update [master]:1

No changes were made, skipping runAfterUpdate.sh.
If you encounter problems, you can start it manually with:
sudo /home/brewpi/utils/runAfterUpdate.sh

The update script can automatically check your controller firmware version and program it with the latest release on GitHub, would you like to do this now? [Y/n]:n
Skipping controller update


*** Done updating BrewPi! ***

Please refresh your browser with ctrl-F5 to make sure it is not showing an old cached version.
root@1c88e2c00c7b:~/brewpi-tools#

As you can see, I never got the option for legacy branch. So then I went ahead and did more digging and found this thread. I copied the code by Elco and ran it as is, from my host location. This is what followed:

pi@bakepi:~ $ docker exec -it brewpi bash
git fetch --unshallow
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin
git checkout legacy 
cd /var/www/html
git fetch --unshallow
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin
git checkout legacy
exitroot@1c88e2c00c7b:/home/brewpi# git fetch --unshallow
remote: Enumerating objects: 1821, done.
remote: Counting objects: 100% (1821/1821), done.
remote: Compressing objects: 100% (660/660), done.
remote: Total 1780 (delta 1176), reused 1717 (delta 1113), pack-reused 0
Receiving objects: 100% (1780/1780), 490.00 KiB | 431.00 KiB/s, done.
Resolving deltas: 100% (1176/1176), completed with 33 local objects.
remote: Enumerating objects: 35, done.
remote: Total 35 (delta 0), reused 0 (delta 0), pack-reused 35
Unpacking objects: 100% (35/35), done.
From https://github.com/BrewPi/brewpi-script
 * [new tag]         0.1.1                                 -> 0.1.1
 * [new tag]         0.2.0                                 -> 0.2.0
 * [new tag]         0.2.0.1                               -> 0.2.0.1
 * [new tag]         0.2.0.2                               -> 0.2.0.2
 * [new tag]         0.2.0.3                               -> 0.2.0.3
 * [new tag]         0.2.0.4                               -> 0.2.0.4
 * [new tag]         0.2.0.4.1                             -> 0.2.0.4.1
 * [new tag]         0.3.0                                 -> 0.3.0
 * [new tag]         0.3.0.1                               -> 0.3.0.1
 * [new tag]         0.3.1                                 -> 0.3.1
 * [new tag]         0.3.1.1                               -> 0.3.1.1
 * [new tag]         0.3.2                                 -> 0.3.2
 * [new tag]         0.3.3.1                               -> 0.3.3.1
 * [new tag]         0.3.4                                 -> 0.3.4
 * [new tag]         0.3.4.1                               -> 0.3.4.1
 * [new tag]         0.3.4.2                               -> 0.3.4.2
 * [new tag]         0.3.4.3                               -> 0.3.4.3
 * [new tag]         0.3.4.4                               -> 0.3.4.4
 * [new tag]         0.3.5                                 -> 0.3.5
 * [new tag]         0.3.6                                 -> 0.3.6
 * [new tag]         0.3.6.1                               -> 0.3.6.1
 * [new tag]         0.3.6.2                               -> 0.3.6.2
 * [new tag]         0.3.6.3                               -> 0.3.6.3
 * [new tag]         0.3.7                                 -> 0.3.7
 * [new tag]         0.3.7.1                               -> 0.3.7.1
 * [new tag]         0.3.8                                 -> 0.3.8
 * [new tag]         0.4.0                                 -> 0.4.0
 * [new tag]         0.4.1                                 -> 0.4.1
 * [new tag]         0.4.2                                 -> 0.4.2
 * [new tag]         0.4.3                                 -> 0.4.3
 * [new tag]         0.4.5                                 -> 0.4.5
 * [new tag]         serial-error-message                  -> serial-error-message
 * [new tag]         skip-settings-restore-without-version -> skip-settings-restore-without-version
 * [new tag]         v0.1                                  -> v0.1
 * [new tag]         wificheck-cron-fix                    -> wificheck-cron-fix
root@1c88e2c00c7b:/home/brewpi# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
root@1c88e2c00c7b:/home/brewpi# git fetch origin
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Total 51 (delta 28), reused 28 (delta 28), pack-reused 23
Unpacking objects: 100% (51/51), done.
From https://github.com/BrewPi/brewpi-script
 * [new branch]      develop                       -> origin/develop
 * [new branch]      feature/port-by-serial-number -> origin/feature/port-by-serial-number
 * [new branch]      legacy                        -> origin/legacy
 * [new branch]      legacy_dev                    -> origin/legacy_dev
 * [new branch]      release/0.5.0                 -> origin/release/0.5.0
root@1c88e2c00c7b:/home/brewpi# git checkout legacy 
error: Your local changes to the following files would be overwritten by checkout:
	settings/config.cfg.example
	settings/defaults.cfg
Please commit your changes or stash them before you switch branches.
Aborting
root@1c88e2c00c7b:/home/brewpi# cd /var/www/html
root@1c88e2c00c7b:/var/www/html# git fetch --unshallow
remote: Enumerating objects: 1877, done.
remote: Counting objects: 100% (1877/1877), done.
remote: Compressing objects: 100% (624/624), done.
remote: Total 1839 (delta 1241), reused 1784 (delta 1186), pack-reused 0
Receiving objects: 100% (1839/1839), 561.74 KiB | 429.00 KiB/s, done.
Resolving deltas: 100% (1241/1241), completed with 31 local objects.
remote: Enumerating objects: 6, done.
remote: Total 6 (delta 0), reused 0 (delta 0), pack-reused 6
Unpacking objects: 100% (6/6), done.
From https://github.com/BrewPi/brewpi-www
 * [new tag]         0.2.0      -> 0.2.0
 * [new tag]         0.3.0      -> 0.3.0
 * [new tag]         0.3.1      -> 0.3.1
 * [new tag]         0.3.2      -> 0.3.2
 * [new tag]         0.4.0      -> 0.4.0
 * [new tag]         v0.1       -> v0.1
root@1c88e2c00c7b:/var/www/html# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
root@1c88e2c00c7b:/var/www/html# git fetch origin
remote: Enumerating objects: 30, done.
remote: Counting objects: 100% (30/30), done.
remote: Total 55 (delta 30), reused 30 (delta 30), pack-reused 25
Unpacking objects: 100% (55/55), done.
From https://github.com/BrewPi/brewpi-www
 * [new branch]      develop               -> origin/develop
 * [new branch]      feature/chart-export  -> origin/feature/chart-export
 * [new branch]      feature/notifications -> origin/feature/notifications
 * [new branch]      legacy                -> origin/legacy
root@1c88e2c00c7b:/var/www/html# git checkout legacy
D	data/.gitignore
D	data/Sample Data/Sample Data-2012-09-26.json
D	data/Sample Data/Sample Data.csv
D	data/profiles/Sample Profile.csv
Branch legacy set up to track remote branch legacy from origin.
Switched to a new branch 'legacy'
root@1c88e2c00c7b:/var/www/html# exit
exit

It seemed like everything worked. So I launched brewPi on my web-browser.

The GUI shows up - says Script Running but I get this:
Cannot receiveLCD text fromPython script

The graph does not work Could not parse data for this brew.
Under logs I get:
Received invalid message on socket: lcd

If I go to the Maintenance Panel, under Device Configuration nothing shows up. If I refresh it gives me this message
Error while receiving device configuration: SyntaxError: Unexpected token C in JSON at position 0

I am unsure if the Rpi is finding the Arduino. I’m unsure what other issues are going on too.

Any help would be greatly appreciated, I’m aware this is not officially supported and I truly appreciate any help I could get.

Thank you in advance!

Elco · May 23, 2019, 1:15pm

This will happen if you are not running the legacy branch. Did you try these instructions?

citybaker · May 23, 2019, 1:50pm

Yeah those were the exact instructions I followed. I’ve pasted the outcome of that in the previous reply as well. It seemed to work as posted above.
I think I might be running these instructions from an incorrect location? I’m confused about what “rootXYZ” is as opposed to “bakepi” which is my host.

Elco · May 23, 2019, 2:23pm

Ah sorry, I wasn’t paying attention.
The instructions didn’t work because you had changes that would be lost by the checkout.
Please run git stash to undo the changes before running git checkout legacy

citybaker · May 23, 2019, 2:58pm

No worries! Thanks! I will give that a shot tomorrow when I’m with my setup.

Meanwhile, I could use a little clarification.
Some questions about docker and git in general for anyone who can answer -
My confusion is about what lives where and if anything gets copied on the host or it lives online? Also what does it mean when I “checkout” a branch of the code? Where is it getting installed? Also, once I checkout the legacy branch, does that become a part of the container on my host RPi?
Sorry if these are basic. I’ve never used git or any of these things for that matter but still keen on learning.

Thanks!

Bob_Steers · May 23, 2019, 4:07pm

Git is a Version Control System. Its purpose is to keep track of changes people made to files in a specific directory. You typically have Local and Remote versions of a repository (the directory containing tracked files). The Local repository lives on your computer, the Remote somewhere else (in this case: GitHub).

git checkout legacy is like loading the saved file “legacy”: it will set all files in your directory to how they are specified in the saved state. You can then either directly run these files, or use them to build a new Docker Image. Your Docker Image will not be automatically changed when you use git checkout.

citybaker · May 24, 2019, 5:51am

Thanks for the info. That does help.

So I followed the instructions, and it worked reverting to the legacy build but now there is a new issue.

The Script is not running. If I click on the button for the Script Start/Stop, nothing happens.
And in the logs I get:
Error: Received invalid message on socket: getTemperatures

FYI, right now I just have one temperature sensor connected to the arduino and it is being recognized in the Device list. So I’m not sure what that error is.

Any thoughts?

citybaker · May 24, 2019, 6:05am

Additionally, I looked through some other posts here that had the similar problem so I ran fixPermissions.sh and got this:

root@4c8c7ffc5383:/home/brewpi# sudo bash /home/brewpi/utils/fixPermissions.sh

***** Fixing file permissions for /var/www *****

***** Fixing file permissions for /home/brewpi *****

chown: invalid user: ‘brewpi:brewpi’

*** ERROR ERROR ERROR ERROR ERROR ***

----------------------------------

See above lines for error message

Script NOT completed

Bob_Steers · May 24, 2019, 6:27am

chown dir username sets the ownership of a directory.

You’re in a root shell. You’ll likely want to exit that before running the script.

citybaker · May 24, 2019, 6:39am

Ah okay. Could you please elaborate on that?

What should I be doing from /home/brewpi ?

Bob_Steers · May 24, 2019, 6:49am

The script likely uses current user (brewpi). You’re logged in as the superuser (can do everything) account “root”. This may mess with some scripts that assume you’re the normal user.

May want to google “root shell”, or “exit root terminal” for explanations.

citybaker · May 24, 2019, 7:14am

I’m sorry but I’m not sure I understand.
I have ssh’d into the docker container ‘brewpi’ so that I can run the script for fixPermissions.sh
I am running it from /home/brewpi/

I’m not sure what to do with chown dir username. What user name do I use? I don’t recall setting any, other than for brewpi port81.

Bob_Steers · May 24, 2019, 7:34am

Close and restart your ssh session, but now prepend the address with brewpi@. ssh address becomes ssh brewpi@address

Elco · May 24, 2019, 7:45am

Forget what Bob said and forget about fixing permissions. Bob didn’t realize you were running things inside the docker container, where things are running as root. Bob knows everything about our new BrewBlox system, which he built. This is the old stuff were things are not as nicely built.

You have switched the brewpi-script repository to legacy, but have not yet switched the brewpi-www repository. You are getting an error because the versions don’t match.

Enter the container with:

docker exec -it brewpi /bin/bash

Then run

cd /var/www/html
git stash
git checkout legacy

citybaker · May 24, 2019, 7:48am

Ah I see. Thanks for the clarification.

Elco,
I’m pretty sure both the dirs have been successfully changed to legacy. I’m still finding that the script is not running.

Elco · May 24, 2019, 7:51am

If you are sure, force refresh the UI (ctrl-F5) or open it in a private window.
Or run git status in both dirs to check that you have succeeded.

Note that by choosing a DIY build based on software we deprecated in 2015, you’re not taking the easy path. We don’t fix any bugs, add any features or offer any support for the discontinued Arduino version.

citybaker · May 24, 2019, 8:09am

Yes I completely understand and looking into a proper solution. I appreciate the help really.

I had to reboot the pi. I did as per your instructions.
First I checked git status on home/brewpi and var/www/html/ and this is what I got:

  root@4c8c7ffc5383:/var/www/html# git status
    On branch legacy
    Your branch is up-to-date with 'origin/legacy'.
    Changes not staged for commit:
      (use "git add/rm <file>..." to update what will be committed)
      (use "git checkout -- <file>..." to discard changes in working directory)

	deleted:    data/.gitignore
	deleted:    data/Sample Data/Sample Data-2012-09-26.json
	deleted:    data/Sample Data/Sample Data.csv
	deleted:    data/profiles/Sample Profile.csv

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	data

no changes added to commit (use "git add" and/or "git commit -a")
root@4c8c7ffc5383:/var/www/html# cd ..
root@4c8c7ffc5383:/var/www# cd ..
root@4c8c7ffc5383:/var# cd ..
root@4c8c7ffc5383:/# cd home/brewpi/data
root@4c8c7ffc5383:/home/brewpi/data# git status
fatal: Not a git repository (or any parent up to mount point /data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
root@4c8c7ffc5383:/home/brewpi/data# cd ..
root@4c8c7ffc5383:/home/brewpi# git status
On branch legacy
Your branch is up-to-date with 'origin/legacy'.
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	deleted:    data/.gitignore
	deleted:    data/Sample Data/Sample Data-2012-09-26.json
	deleted:    data/Sample Data/Sample Data.csv
	deleted:    logs/.gitignore
	deleted:    settings/.gitignore
	deleted:    settings/config.cfg.example
	deleted:    settings/defaults.cfg
	deleted:    settings/tempProfile.csv

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	data
	logs
	settings

no changes added to commit (use "git add" and/or "git commit -a")

Note that git stash is yielding me these lines:

root@4c8c7ffc5383:/home/brewpi# git stash
error: 'data/.gitignore' is beyond a symbolic link
fatal: Unable to process path data/.gitignore
Cannot save the current worktree state

On the webserver, I’m still at Script not running and the logs are now saying:

Traceback (most recent call last):
File "/home/brewpi/brewpi.py", line 183, in 
dontRunFilePath = os.path.join(config['wwwPath'], 'do_not_run_brewpi')
File "/usr/local/lib/python2.7/dist-packages/configobj.py", line 554, in __getitem__
val = dict.__getitem__(self, key)
KeyError: 'wwwPath'

Device Configuration now yields nothing, no devices are listed now.
The device config log says:

Error while receiving device configuration: SyntaxError: Unexpected token C in JSON at position 0

Bob_Steers · May 24, 2019, 8:52am

Your /home/brewpi directory is up to date with legacy, so you can ignore the git stash fail for now. Next step is to follow Elco’s code to checkout the legacy version in /var/www/html.