I have my brewlox installed on an ubuntu machine and I run a bunch of other docker images other than brewblox on a range of ports so I can co-host them.
I started looking into Traefik and I would like to us it for other use cases, so I have a few options:
- start my own instance seperate to BrewBlox, modify BrewBlox to suite
- make my other instances hook into the BrewBlox instance
- Configure another IP and bind a seperate instance to that, keeping them seperate.
I haven’t found any documentation on how to do 3, the ability to scope/assign docker labels to a given Traefik instance.
So then, the next choice is probably based on how the docker-compose.shared.yml is maintained.
I know I could probably remove the traefik service from the shared yaml, but will this get overwritten with the next brewblox update?
I ideally I would like to run my own Traefik and have brewblox hook into it via it’s labels.
Do you want your other images to share ports with Brewblox?
Do you want Traefik to handle SSL certs?
I would not recommend option 3. It would not solve your problems, and create a whole set of new ones.
We do overwrite the shared.yml every update, but you can override it in the docker-compose.yml file.
Important context is that Traefik migrated to a new configuration syntax. We will adopt the new syntax pretty soon: we figured out required changes already, and are now just waiting for a suitable release to add it to.
Scoping is part of these changes. I think that the most elegant approach would be to override the Brewblox services to use the new, scoped, configuration.
You can then use separate Traefik runtimes for Brewblox and your other containers, or have them share Traefik, and configure ports/routing with labels.
I’ll look up my implementation notes for Traefik config, and do a quick write up tomorrow or so.
Yeah, ideally I would just seperate them by DNS names brewblox.home, other-thing.home using traefik.frontend.rule=Host: rules.
How are my docker-compose and the docker-compose.shared combined? Been looking for the code that does that. So far the best I can do is https://github.com/BrewBlox/brewblox-ctl-lib/search?q=read_shared_compose&unscoped_q=read_shared_compose
I’ll look up my implementation notes for Traefik config, and do a quick write up tomorrow or so.
Awesome, thanks 
It’s part of docker-compose itself, but we do override the used file names by setting COMPOSE_FILE in .env
It looks like I can “disable” the packaged Traefik by the following in my docker-compose.yml
services:
traefik:
image: tianon/true # Super small image
command: "true"
entrypoint: "true"
And also move the ports that it binds to by updating .env
BREWBLOX_PORT_HTTP=40080
BREWBLOX_PORT_HTTPS=40443
This would allow me to standup my instance on the well known ports.
Here you go. This was surprisingly fun to set up. We may adopt the concept of publishing localhost as brewblox.local to reduce the hassle with IP addresses.
Known constraint: the http -> https redirect will also be active for your services.
You can configure this by tweaking routing between web and websecure entrypoints, but better take one step at a time.
The implementation still places the traefik container in the Brewblox project.
This can be changed, but may cause some issues with brewblox-ctl update. I haven’t tried out all edge cases there.
Edit: Guide is now available at https://brewblox.netlify.app/dev/tutorials/subrouting.html
This is awesome. I’ll play tomorrow.
Cheers!
Is this also new?
eventbus:
image: brewblox/mosquitto:${BREWBLOX_RELEASE}
labels:
- traefik.http.services.eventbus.loadbalancer.server.port=15675
My current eventbus configuration is
eventbus:
image: brewblox/rabbitmq:${BREWBLOX_RELEASE}
restart: unless-stopped
labels:
- "traefik.port=15675"
- "traefik.protocol=ws"
- "traefik.frontend.rule=PathPrefix: /eventbus"
So websockets access. Just want to make sure this isn’t leaking the new release
Adam
You can leave out the image if you want: we configured RabbitMQ and Mosquitto to use the same port for MQTT over websockets.
Traefik v2 is smart enough to automatically proxy WS requests, so no need for explicit configuration there.
Had a successful night, my brewblox is part of an external traefik instance and accessible from the public internet (Google OAuth protected). I wanted brewblox to join an external one as I didn’t want updates to brewblox to impact my other running services.
I ended up just changing docker-compose.shared.yml to get it going, tomorrow night I will look to clean it up and see if I can drive it entirely from docker-compose.yml
Once clean, I will show you what I needed to do, to see if there are any better ways to represent this as so I can have the least amount of impact with brewblox updates.
Thanks for all the guides, the prefix-strip middleware would have had me going for a while with datastore if I didn’t know about it upfront.
1 Like
Today’s release includes the migrate to Traefik v2.
For you, this means you don’t need to override the datastore / history / eventbus services.
I updated the guide I posted earlier, and it’s now available at https://brewblox.netlify.app/dev/tutorials/subrouting.html
Just updated the stack with the new release which definitely made it easier and cleaner.
I made the following additions to my external traefik docker-compose.yml
networks:
...
brewblox:
external:
name: brewblox
..
traefik:
networks:
- brewblox
the following changes to my .env
DOMAIN_NAME=mydomain.com
COMPOSE_PROJECT_NAME=brewblox
BREWBLOX_PORT_HTTP=4080
BREWBLOX_PORT_HTTPS=40443
And this is my docker-compose.yml for brewblox
version: '3.7'
networks:
default:
driver: bridge
external:
name: brewblox
services:
brewpi:
command: --name=brewpi --mdns-port=${BREWBLOX_PORT_MDNS} --discovery=all --device-id=40002f001847383434353030
image: brewblox/brewblox-devcon-spark:${BREWBLOX_RELEASE}
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.brewpi.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/brewpi`)
- traefik.http.routers.brewpi.tls=true
- traefik.http.routers.eventbus.middlewares=chain-oauth@file
privileged: true
restart: unless-stopped
datastore:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.datastore.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/datastore`)
- traefik.http.routers.datastore.tls=true
- traefik.http.routers.datastore.middlewares=prefix-strip,chain-oauth@file
eventbus:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.eventbus.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/eventbus`)
- traefik.http.routers.eventbus.tls=true
- traefik.http.routers.eventbus.middlewares=prefix-strip,chain-oauth@file
ports:
- 5672:5672
history:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.history.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/history`)
- traefik.http.routers.history.tls=true
- traefik.http.routers.history.middlewares=chain-oauth@file
traefik:
entrypoint: /true
image: tianon/true
restart: 'no'
ui:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.ui.rule=Host(`brewblox.$DOMAIN_NAME`) && (PathPrefix(`/ui`)
|| Path(`/`))
- traefik.http.routers.ui.tls=true
- traefik.http.routers.ui.middlewares=chain-oauth@file
The extra traefik labels (enable, tls, chain-oauth middlewares) are to work with my setup.
The docker-compose.share.yml is vanilla as of the the current version.
It all works well!
The only issue that is that when the update script runs and is verifying services is that it attempts to goto https://localhost:40443 which is not started, and even if it went through the current entrypoint it wouldn’t pass authentication. I might work on a simple nginx reverse proxy that doesn’t have all the authentication and bind it to localhost:40443.
That looks like a pretty clean setup!
I do spot eventbus having a 5672:5672 port mapping. That is the AMQP port from RabbitMQ. MQTT/Mosquitto doesn’t use it.
If you want to expose the MQTT broker, you can change it to 1883:1883, otherwise you can remove it.
You can repurpose the traefik service for this. If you configure it to only use a file provider, it will ignore the label-based configuration.
Edit: you can also skip the service migrate by using brewblox-ctl update --no-migrate. It’ll also skip any automatic migrations we’re performing, but that’s probably not a bad thing with your custom setup.
The eventbus is for a tilt service running on a RaspberryPi in my fridge, which I have to admit, I haven’t tested yet.
Do I still need the --mdns-port=${BREWBLOX_PORT_MDNS} on the brewpi command? Looking at the doco for the latest release, it seem like it can be removed.
I’ll look into reusing traefik, I thought it might just work out of the box, given it it is on different ports and the constraint
--providers.docker.constraints="LabelRegex(`com.docker.compose.project`, `${COMPOSE_PROJECT_NAME}`)"
I even tried add localhost into the rule
- traefik.http.routers.brewpi.rule=(Host(`localhost`) || Host(`brewblox.$DOMAIN_NAME`)) && PathPrefix(`/brewpi`)
- traefik.http.routers.datastore.rule=(Host(`localhost`) || Host(`brewblox.$DOMAIN_NAME`)) && PathPrefix(`/datastore`)
- traefik.http.routers.eventbus.rule=(Host(`localhost`) || Host(`brewblox.$DOMAIN_NAME`)) && PathPrefix(`/eventbus`)
- traefik.http.routers.history.rule=(Host(`localhost`) || Host(`brewblox.$DOMAIN_NAME`)) && PathPrefix(`/history`)
- traefik.http.routers.ui.rule=(Host(`localhost`) || Host(`brewblox.$DOMAIN_NAME`)) && (PathPrefix(`/ui`)
localhost shows up in my main traefik instance, but given that it’s all auth’d it’s ok.
But 404s with curl --insecure https://localhost:40443/datatore
I’ll keep looking into it.
The Tilt by default uses MQTT over websockets (targets HOST:443/eventbus). I suspect that OAuth breaks that, and that the easiest solution is to open up 1883, and add --mqtt-protocol=mqtt to the tilt args.
You can remove the --mdns-port argument.
datatore -> datastore. Not sure whether that’s a typo in your request, or just in the forum post.
There can also be multiple Router / Middleware / Service combos for each docker-compose service.
Untested, but I suspect that this may work:
labels:
...
- traefik.http.routers.internal-datastore.rule=Host(`localhost`) && PathPrefix(`/datastore`)
- traefik.http.routers.internal-datastore.middlewares=prefix-strip
That works perfectly.
My final configuration
External Traefik docker-compose.yml
networks:
...
brewblox:
external:
name: brewblox
..
traefik:
networks:
- brewblox
brewblox .env
DOMAIN_NAME=mydomain.com
COMPOSE_PROJECT_NAME=brewblox
BREWBLOX_PORT_HTTP=4080
BREWBLOX_PORT_HTTPS=40443
brewblox docker-compose.yml
networks:
default:
driver: bridge
external:
name: brewblox
services:
brewpi:
command: --name=brewpi --discovery=all --device-id=40002f001847383434353030
image: brewblox/brewblox-devcon-spark:${BREWBLOX_RELEASE}
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.brewpi.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/brewpi`)
- traefik.http.routers.brewpi.tls=true
- traefik.http.routers.brewpi.middlewares=chain-oauth@file
- traefik.http.routers.internal-brewpi.rule=Host(`localhost`) && PathPrefix(`/brewpi`)
privileged: true
restart: unless-stopped
datastore:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.datastore.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/datastore`)
- traefik.http.routers.datastore.tls=true
- traefik.http.routers.datastore.middlewares=prefix-strip,chain-oauth@file
- traefik.http.routers.internal-datastore.rule=Host(`localhost`) && PathPrefix(`/datastore`)
- traefik.http.routers.internal-datastore.middlewares=prefix-strip
eventbus:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.eventbus.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/eventbus`)
- traefik.http.routers.eventbus.tls=true
- traefik.http.routers.eventbus.middlewares=prefix-strip,chain-oauth@file
- traefik.http.routers.internal-eventbus.rule=Host(`localhost`) && PathPrefix(`/eventbus`)
- traefik.http.routers.internal-eventbus.middlewares=prefix-strip
ports:
- 1883:1883
history:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.history.rule=Host(`brewblox.$DOMAIN_NAME`) && PathPrefix(`/history`)
- traefik.http.routers.history.tls=true
- traefik.http.routers.history.middlewares=chain-oauth@file
- traefik.http.routers.internal-history.rule=Host(`localhost`) && PathPrefix(`/history`)
ui:
labels:
- traefik.enable=true
- traefik.docker.network=brewblox
- traefik.http.routers.ui.rule=Host(`brewblox.$DOMAIN_NAME`) && (PathPrefix(`/ui`)
|| Path(`/`))
- traefik.http.routers.ui.tls=true
- traefik.http.routers.ui.middlewares=chain-oauth@file
- traefik.http.routers.internal-ui.rule=Host(`localhost`) && (PathPrefix(`/ui`)
|| Path(`/`))
version: '3.7'
no changes to docker-compose.shared.yml, so I assume unless there is major changes to traefik, updates should still work. As shown below:
$ brewblox-ctl update
Do you want to remove old docker images to free disk space? [Y/n]: y
Command is about to: Download and apply updates.
Do you want to continue? (yes, no, verbose, dry-run) [press ENTER for default value 'yes']
INFO Updating brewblox-ctl...
INFO Updating brewblox-ctl libs...
edge: Pulling from brewblox/brewblox-ctl-lib
Digest: sha256:3e2e5fcbdccb8aaa7cbb9250499744309ba3ec2f7337f2b010bfd7b3b99ad55c
Status: Image is up to date for brewblox/brewblox-ctl-lib:edge
docker.io/brewblox/brewblox-ctl-lib:edge
46975e3cc1bec9f874858493f1700ae421b6414f22f659840a5f5cae1b1d2e5e
ctl-lib
Command is about to: Download and apply updates.
Do you want to continue? (yes, no, verbose, dry-run) [press ENTER for default value 'yes']yes
INFO Updating configuration files...
INFO Checking Avahi config...
INFO Stopping services...
Removing brewblox_influx_1 ... done
Removing brewblox_traefik_1 ... done
Removing brewblox_ui_1 ... done
Removing brewblox_brewpi_1 ... done
Removing brewblox_datastore_1 ... done
Removing brewblox_history_1 ... done
Removing brewblox_eventbus_1 ... done
Network brewblox is external, skipping
INFO Migrating configuration files...
INFO Checking .env variables...
INFO Pulling docker images...
Pulling influx ... done
Pulling ui ... done
Pulling brewpi ... done
Pulling eventbus ... done
Pulling datastore ... done
Pulling history ... done
Pulling traefik ... done
INFO Starting services...
Creating brewblox_history_1 ... done
Creating brewblox_influx_1 ... done
Creating brewblox_eventbus_1 ... done
Creating brewblox_datastore_1 ... done
Creating brewblox_traefik_1 ... done
Creating brewblox_brewpi_1 ... done
Creating brewblox_ui_1 ... done
INFO Migrating service configuration...
Connecting https://localhost:40443/history/ping, attempt 1/60
Success!
Connecting https://localhost:40443/datastore, attempt 1/60
Connecting https://localhost:40443/datastore, attempt 2/60
Success!
INFO Updating version number to 0.5.0...
INFO Pruning unused images...
Total reclaimed space: 0B
INFO Pruning unused volumes...
Deleted Volumes:
...
Total reclaimed space: 238.9kB
I must say, you have made some great design choices and put together a flexible architecture, I bet you never expected for anyone to try to do this.
Thanks again
1 Like