Import historical data to fermention log

I have recently installed my BrewPi along with a temperature sensor as I wait for my Spark to arrive. I’ve got a brew that I’ve been logging in another application for the last 3 weeks, and I’d like to migrate the data into the BrewPi database. As I understand, the data is stored in .json files, and I’ve been able to open, edit, and see some results in the BrewPi chart. But the data doesn’t seem to be consistent. Within the brewpi/data folder, there are separate folders for each of the beers, and within the beer I’m logging, there are six .json files. Should I create a new .json file for each day logged? or can one file handle 21 days worth of data? Any suggestions on how to best migrate my data?

Many thanks!

You can put all data in one file.
The one file per day format was chosen in the past when we still used google charts to display the data. The API could only write the entire data that it had in memory to a file and could not write to a file incrementally. To limit the data in memory, we started a new file each day. When we later switched to DyGraphs, we kept the format.

I would just write a small python script to read your old file and put it in the BrewPi json file.

Look at this class:



Excellent. I will work on it tonight. Meanwhile, my Spark enclosure and temp probes arrived a few minutes ago from Customs. I’m a happy camper… Can hardly wait to get home!

On the topic of temperature log files @Elco where can I find the code related to exporting the logs as .csv files? I’d like to add in a header and change it to be comma separated so the files can be easily opened in excel.

The files open fine in Excel for me, my Excel prefers semicolon separated. They are missing a header though. Just search for .csv.

thanks @Elco.

When I tried automatically opening the files in excel (double clicking the .csv file) it wouldn’t properly pick up the semicolon separator. You can open excel first and then open the csv file and manually tell it the separator type though. CSV stands for “comma separated values” so commas are probably better.

Yeah, I know. I think it might be related to the system language and regional settings. In Holland, we use commas as a decimal separator and I think that is why Excel defaults to a semicolon for cell separators. I dislike that though, I wish excel would just handle CSVs properly, how they are intended to be used (like you said).

If you send in a pull request for adding a header and switch to commas I’ll gladly merge it.

My export/import strategy is to rename the downloaded .csv file with the .dat file extension, then open with Excel. This causes Excel to load the Text Import Wizard as shown below. Choose Delimited, on the next screen choose Semicolon as the delimiter, then hit Finish. You end up with 9 columns of clean data.

I made the attached Excel spreadsheet in order to graph the beer temp vs. beer setting, and on a secondary axis show the % error in beer temp. If you copy and paste the imported .dat file columns into the table of my spreadsheet then the graph will automatically adjust, along with calculations like “total # of days”, “Average beer temp error”, etc.

Feel free to use / improve: BrewPi temperature error log.xlsx (837.0 KB)

Is the month value in the json data file n-1? For some reason, all my data for April appears as
The month and time on my Pi are working properly. When I imported my historical data from the other application, my graph went crazy with lines waaay off the charts. But when I open the CSV file that is created automatically, the month value is correct. Any explanation?

Yes, months are counted 0-11 in JavaScripts Datetime…
Python uses 1-12, so we subtract 1 before saving as json.