Development Update?


#1

Hey @Elco. It’s been a while since we’ve had any update on how development is going for the complete architecture redesign. You used to give regular updates but I haven’t seen one for a while. Are you able to tell us a bit that’s been happening over the last few months?

Did you ever employ a new Developer to help you out?

I’m particularly interested in a better interface for managing brew day (ie easy switching between mash control and boil control). I realise this isn’t the highest priority though.


#2

Hi Dan,

I don’t have time right now to write a big elaborate blog post, so here’s a quick update

  • I managed to hire a full time developer named Bob, who is working full time on the new stack and making great progress (github.com/brewblox). He’s done his internship at BrewPi for the past 5 months and started on a contract since July. He’s working mainly on the service layer: microservices, controller communication, data logging, rest api.
  • I have also hired a freelance front end developer (Gaya) to work on the Vue frontend for 2 days a week, for the past 5 months.
  • I’m tackling the firmware changes myself.

This has resulted in a lot of progress! The state of things now:

  • The service layer has been rewritten as microservices running in their own docker container, this is an entirely new stack. The services can be queried via REST and there’s an event bus for inter-service communication. We’re making it easy to integrate third party services and hardware too.
  • The UI is being developed as a Vue/Quasar (with Typescript!) dashboard which the user can fully customize by adding blocks. These are a live representation of the hardware and fully customizable charts. Currently Gaya is working on porting the interactive process view from this React prototype (https://brewpi-ui-demo.herokuapp.com/processview/herms-automated-valves). The basics for charts and simple hardware representation blocks are done.
  • The firmware is being rewritten to also be a flexible set of blocks (PIDs, sensors, actuators) that the user can create and link together in any way he pleases.

We’re currently integrating the layers and refactoring where needed for that.

Due to the work on the new stack, the old stack has not seen many updates. It’s not backwards compatible.

We’re trying to have a first alpha version out in September. It’s taking a long time, but then you’re getting everything at once. Multiple processes, customizable charts, mashing layout, etc.


#3

Thanks @Elco. Appreciate the update. Glad to hear you’re making some good progress and looking forward to the alpha release in a few months. I realize it’s a bit task you’re aiming for.


#4

Good news. With the third party integration are you also allowing for third party software to grab the data from the BrewPi? Specifically I was thinking about the new Brewfather software, being software that already has integration with Tilt/iSpindle hydrometers and BrewPiless, it seems to be really helpful to see all the data in one place for an ongoing brew and as it is cloud based and passive less prone to security issues.


#5

Yes, all services have a rest interface. But writing a new container to integrate the Tilt into our system is probably less work than the other way around.


#6

I’m very much looking forward to this - I am finally getting my control box done this weekend (is the plan). I got my shelf with all the valves and pumps built. Get the hoses all measured and hooked up today. I just need to do my volume measurement markings in the kettles. :grinning: Then I will submit some pictures and stuff for the group when its all setup.


#7

Is a September release a pretty sure thing or should I be looking at a second Spark for a second fermenter?


#8

@Elco Just wanted to follow up with you and your time on progress / status of the “new” development update you guys have been working on? Looking forward to it - thanks
Ill add pictures of my setup after the control box is all done :smiley:


#9

During my first week of holiday I did a bit of groundwork refactoring the new framework (laptop on the beach, I get bored quickly and prefer coding puzzles to sudoku :slight_smile: ). In the second week, I read 4 of Scott Meyers C++ books which gave me a lot of inspiration and I got excited about using the new C++14 features and much more of the STL.

So when I came back, I refactored the entire framework we use for managing objects on the controller (like sensors, pids, actuators, etc). I’m very happy with the new implementation and have 97% test coverage for it now!

If you are curious, look here (readme in repo is outdated),

This is the right moment to make some fundamental changes to how the control system works, so I’m not only switching to a new framework, but also change how objects reference each other and how calculations are done.

Last week I spent integrating a new fixed point library called CNL and I integrated a new filtering implementation I coded earlier which gives a much flatter passband, less delay and has step detection. This enables wider temperature ranges in the future, so we can add support for higher temperature sensors for example. I just refactored the PID class and my next goal is refactoring the actuators to simplify constraints (min/max, time limits, mutex, load balancing).

So done for firmware:

  • Framework for communcation, object management, settings storage.
  • Sending/receiving object data with protocol buffers
  • Refactoring of temperature sensors, setpoint, PID, mock objects

Todo:

  • Refactor actuators
  • New display code
  • More unit tests
  • More integration tests

See the commit log here. I’m a bit disappointed that a release in September is unlikely but the pace is great and I do expect to have a demo ready in the first half of October. We’ll start testing with a few selected testers and if that goes well roll it out for others too.

Meanwhile, Bob and Gaya have done a ton of work on the service layer and UI. The framework for dashboards and widgets is done. We have live updating charts, dashboard widgets and setup wizards. Most of the work to be done is implementing the actual widgets for the different block types (Pid, Setpoint, etc).
You can follow that development here.

I’m very sorry that we’ll overrun my estimate of September, but not by much. I’m 100% certain the new feature set will blow you away though.


#10

Thanks, this is a really great update! While I have been following the github commits over the summer, it’s nice to get a big picture update with editorial comments every once in a while.

Definitely looking forward to the alpha in October. While I won’t lie about it being hard to wait, I’m sure down the line it will be worth it having all the code well designed and as generic/mod-able as possible.


#11

Any info available on how other projects can hook into the new software? For example ispindle being able to push info to brewblox?


#12

We have boilerplate for writing your own docker containers that can hook into the platform.

There is a global RabbitMQ eventbus for generated data. If the iSpindle container pushes data onto that bus, it will automatically be logged by the history service and will be available to graphs in the UI.

For reading/writing data and settings to the container directly, it will provide its own REST API and we also have boilerplate for that.

So to integrate the iSpindle, you will have to write I think max 100 lines of code to handle communicating with it.
And perhaps you’ll want to add a new widget to use on the UI dashboards.

For a demo, Bob integrated an Xbox controller and displayed button state in Graphs. That didn’t require much code:

So to summarize:
We have a basic set of containers that provide the framework for data logging and managing the containers. It will be very easy to add a new container and it will run completely independent of the others but it can be integrated using the provided communication channels. There are no code dependencies.


#13

Its now mid-late October…


#14

Hi @matt_m,

I’m well aware of that. We’re working our asses off to deliver as soon as possible.
As you can see here:
https://github.com/BrewPi/firmware/commits/feature/brewblox
And here:

Latest developments:

  • Almost all control objects are ported to the new framework. Only some OneWire actuators are left
  • The firmware got some new exciting features:
    • All digital actuators will have a user configurable set of constraints. Minimum off time, minimum on time and/or mutually exclusive with other actuators.
    • All analog actuators (this includes PWM) will have a list of constraints, this includes min, max and load balancing
    • The constraints above can be used for example to run your HLT and your BK heating element at the same time when your fuses only support one at a time. The high cycle of one PWM will overlap with the low cycle of the other (mutex) and the load balancer will ensure that the sum of both doesn’t exceed 100%.
  • Temperature profiles have moved to run on the Spark instead of the server
  • PID and PWM classes were almost completely rewritten for better performance and reliability
  • Digital filtering has completely changed for better filtering with less delay

On the UI side:

  • UI got user configurable units (Celsius/Fahrenheit/etc)
  • Charts can be user configured for which data to show and over which period
  • Chart data is now streamed with SSE to be displayed live
  • New menus to create the constraints
  • New configuration wizards
  • Reordering of dashboards is possible

We have reached end-to-end and are ready to test on hardware. We will start doing this in the coming week with a select group of test customers. Things are moving forward fast, but at the same time it just a lot of new code. We’re working very hard. Check the commit logs and confirm for yourself.

We want to release a public beta as soon as the minimum set of requirements to run a brewery is met, but not sooner.


#15

Well Done @Elco and team, this looks great, cannot wait for the public beta.