Pilot lights as indicators

I want to add two pilot lights as indicators to what the BrewPi is doing.
I want to add a red for heater is running, and a blue one for the cooler function.
I planned on mounting them inside a project box and mounting that to the wall.
Can I just wire off the respective ssr and have it work like that?

Yes. Just make sure you buy pilot lights for the appropriate voltage.

So another question I have is can I put both SSRs in a project box? Or would they need airflow around them? I currently have them mounted to the back of my freezer and would like them more out of the way.

They don’t generate much heat when switching a freezer, because the current is relatively low compared to their nominal current.

Maybe you can use a metal (aluminum) project box? You can then just screw them to the bottom of the box, without a heatsink.

I’ve used illuminated switches, so I’m able to manually turn off the heating or cooling. (i.e. if your crash cooling, you don’t necessarily care if you overshoot a bit/do not want the heater to come on).
Switches illuminate only when heating or cooling circuit is activate.

This is a basic arduino using a dual 10a relay board.

I used a selection of these led indicators to track activity. They are visible during the day with no troubles and have been working fine for about a year.
https://cpc.farnell.com/search?st=led%20240v%2022mm

Apologies if the link falls foul of any rules.