Re: Docker setup

[wvengen]

On 19-08-17 17:50, paroga [via foodsoft] wrote:

wvengen wrote

>     wvengen wrote
>           o database setup
>           o redis setup (e.g. manual Docker container exposed in
>     Foodsoft env?)
>
> In our setup in Austria that's just adding the offical Docker images
> to the docker-compose.yml. I can share our repository with whoever
> will setup Docker. Unfortunately it still contains some private data,
> which is the reason why I can not publish it on GitHub ATM.
I'm not very experienced in using Docker for production, so have not
much clue how deployment using plain Docker, docker-compose, ... would
look like in practice. How do security updates work (does that need
updates on docker hub, or would you upgrade system packages within the
docker image)? We could either use a single redis and database docker
container for all setups, or put it in docker-compose (I suppose the
latter would be more automated and hopefully less error-prone and
configuration work).

Doing system updates inside a container is the complete wrong way. You ALWAYS regenerate the whole container. Building is usually automated, e.g. on Docker hub.

I'd recommend using docker-compose! Then you do it the Docker way and you can reuse the existing and offical images. I can setup a minimal setup for the global instance, so you have something to start and play with it.

Okay, that's clear :)
Sounds good. Would be great to have docker-compose auto-deploy on repository change. Do you have that running?

wvengen wrote

>     wvengen wrote
>           o ssl setup with letsencrypt
>
> In our setup in Austria we use a haproxy Docker container in front of
> the foodsoft Docker container. The certificates are generated in a
> seperate container with certbot and are shared with the haproxy via a
> volume.
Sounds like a good idea to do this in a separate container.

I can add this to the sample docker-compose too, if there is an interest.

I would be interested. What do others think, is this a good setup?

wvengen wrote

>     wvengen wrote
>           o figure out how we will manage app_config.yml
>
> In our setup in Austria we build our own "private" foodsoft image
> based on "public" image (published on Docker hub), which just adds the
> app_config.yml via COPY to the image.
> docker-compose.yml and the app_config.yml are managed via git.
Does it make sense to use docker volumes for this? Git history on
app_config (and env?) would be nice (but repo's need to be private -
indeed, @Github or locally on server perhaps).

app_config.yml is a configuration file, which should be version controlled. So a Docker volume is the wrong way IMHO. What kind of sensitive data do you see in the app_config.yml? In Austria we pass our database passwords via environment variables. So no sensitive data is stored in the file.

This brings up a new question: how do we manage database credentials for each foodcoop? I currently give each coop its own database user for security. So then we'd generate a new environment variable with a password for each new coop.

There are also mail and physical addresses in the config file, which might be privacy sensitive. Because these can be set in the admin interface, it could be omitted in app_config. (Though I do like to have contact details from all foodcoops, perhaps they can be stored elsewhere/otherwise.) I do like to make as much as possible publically viewable :)

- Willem

[paroga]

wvengen wrote

On 19-08-17 17:50, paroga [via foodsoft] wrote:
>
>     wvengen wrote
>     >     wvengen wrote
>     >           o database setup
>     >           o redis setup (e.g. manual Docker container exposed in
>     >     Foodsoft env?)
>     >
>     > In our setup in Austria that's just adding the offical Docker
>     images
>     > to the docker-compose.yml. I can share our repository with whoever
>     > will setup Docker. Unfortunately it still contains some private
>     data,
>     > which is the reason why I can not publish it on GitHub ATM.
>     I'm not very experienced in using Docker for production, so have not
>     much clue how deployment using plain Docker, docker-compose, ...
>     would
>     look like in practice. How do security updates work (does that need
>     updates on docker hub, or would you upgrade system packages within
>     the
>     docker image)? We could either use a single redis and database docker
>     container for all setups, or put it in docker-compose (I suppose the
>     latter would be more automated and hopefully less error-prone and
>     configuration work).
>
> Doing system updates inside a container is the complete wrong way. You
> ALWAYS regenerate the whole container. Building is usually automated,
> e.g. on Docker hub.
>
> I'd recommend using docker-compose! Then you do it the Docker way and
> you can reuse the existing and offical images. I can setup a minimal
> setup for the global instance, so you have something to start and play
> with it.

Okay, that's clear :)
Sounds good. Would be great to have docker-compose auto-deploy on
repository change. Do you have that running?

No, I didn't have the time to set it up until now.

wvengen wrote

>     wvengen wrote
>     >     wvengen wrote
>     >           o ssl setup with letsencrypt
>     >
>     > In our setup in Austria we use a haproxy Docker container in
>     front of
>     > the foodsoft Docker container. The certificates are generated in a
>     > seperate container with certbot and are shared with the haproxy
>     via a
>     > volume.
>     Sounds like a good idea to do this in a separate container.
>
> I can add this to the sample docker-compose too, if there is an interest.
I would be interested. What do others think, is this a good setup?

I created a small demo project (didn't test it) to show the basic idea of how we do it in Austria: https://github.com/paroga/foodcoops.net

wvengen wrote

>     wvengen wrote
>     >     wvengen wrote
>     >           o figure out how we will manage app_config.yml
>     >
>     > In our setup in Austria we build our own "private" foodsoft image
>     > based on "public" image (published on Docker hub), which just
>     adds the
>     > app_config.yml via COPY to the image.
>     > docker-compose.yml and the app_config.yml are managed via git.
>     Does it make sense to use docker volumes for this? Git history on
>     app_config (and env?) would be nice (but repo's need to be private -
>     indeed, @Github or locally on server perhaps).
>
> app_config.yml is a configuration file, which should be version
> controlled. So a Docker volume is the wrong way IMHO. What kind of
> sensitive data do you see in the app_config.yml? In Austria we pass
> our database passwords via environment variables. So no sensitive data
> is stored in the file.
This brings up a new question: how do we manage database credentials for
each foodcoop? I currently give each coop its own database user for
security. So then we'd generate a new environment variable with a
password for each new coop.

Do we need different passwords? I see the benefit of having different users to prevent switching the database via an SQL injection, but what additional security do we get out of different passwords?

wvengen wrote

There are also mail and physical addresses in the config file, which
might be privacy sensitive. Because these can be set in the admin
interface, it could be omitted in app_config. (Though I do like to have
contact details from all foodcoops, perhaps they can be stored
elsewhere/otherwise.) I do like to make as much as possible publically
viewable :)

I'd keep app_config.yml as small as possible. Foodcoops move to different locations, change their contact information and maybe someone will update it in the admin interface, but nobody will updated that information in the app_config.yml, since there in no functional difference. So I would avoid it completely.

[wvengen]

On 20-08-17 21:50, paroga [via foodsoft] wrote:

wvengen wrote

On 19-08-17 17:50, paroga [via foodsoft] wrote:

Would be great to have docker-compose auto-deploy on
repository change. Do you have that running?

No, I didn't have the time to set it up until now.

Ok, this could be part of https://github.com/foodcoops/foodcoops.net/issues/5

wvengen wrote

>     wvengen wrote
>     >     wvengen wrote
>     >           o ssl setup with letsencrypt
>     >
>     > In our setup in Austria we use a haproxy Docker container in front of
>     > the foodsoft Docker container. The certificates are generated in a
>     > seperate container with certbot and are shared with the haproxy via a volume
>     Sounds like a good idea to do this in a separate container.
>
> I can add this to the sample docker-compose too, if there is an interest.
I would be interested. What do others think, is this a good setup?

I created a small demo project (didn't test it) to show the basic idea of how we do it in Austria: https://github.com/paroga/foodcoops.net

Awesome, I've forked it to foodcoops (letsencrypt pending - https://github.com/foodcoops/foodcoops.net/issues/1 ).
Certificates should definitely on a volume. If you use a different container, I suppose you're forwarding .well-known to that in haproxy?

wvengen wrote

>     wvengen wrote
>     >     wvengen wrote
>     >           o figure out how we will manage app_config.yml
>     >
>     > In our setup in Austria we build our own "private" foodsoft image
>     > based on "public" image (published on Docker hub), which just
>     adds the
>     > app_config.yml via COPY to the image.
>     > docker-compose.yml and the app_config.yml are managed via git.
>     Does it make sense to use docker volumes for this? Git history on
>     app_config (and env?) would be nice (but repo's need to be private -
>     indeed, @Github or locally on server perhaps).
>
> app_config.yml is a configuration file, which should be version
> controlled. So a Docker volume is the wrong way IMHO. What kind of
> sensitive data do you see in the app_config.yml? In Austria we pass
> our database passwords via environment variables. So no sensitive data
> is stored in the file.
This brings up a new question: how do we manage database credentials for
each foodcoop? I currently give each coop its own database user for
security. So then we'd generate a new environment variable with a
password for each new coop.

Do we need different passwords? I see the benefit of having different users to prevent switching the database via an SQL injection, but what additional security do we get out of different passwords?

Good point. I would even consider one account per instance (could even get rid of the db reconnect call when selecting a different foodcoop if db details don't change), what do you think? I would like to give foodcoops only read access to the sharedlists databases, so sharedlists should certainly have different connection details (and preferably a different password, but that's a different container anyway).

wvengen wrote

There are also mail and physical addresses in the config file, which
might be privacy sensitive. Because these can be set in the admin
interface, it could be omitted in app_config. (Though I do like to have
contact details from all foodcoops, perhaps they can be stored
elsewhere/otherwise.) I do like to make as much as possible publically
viewable :)

I'd keep app_config.yml as small as possible. Foodcoops move to different locations, change their contact information and maybe someone will update it in the admin interface, but nobody will updated that information in the app_config.yml, since there in no functional difference. So I would avoid it completely.

Agree. Then we can make app_config.yml public as well.

I'm happy that step by step there's more clarity on what needs to be done (at least for me).
Please see the first milestone in the foodcoops.net repo - https://github.com/foodcoops/foodcoops.net/milestone/1
Anything important missing?
Next step would be to work on this collectively!

- Willem

[before]

Looks promising. Where do you need help?

[wvengen]

On 12-09-17 18:47, before [via foodsoft] wrote:
> Looks promising. Where do you need help?
All of those in https://github.com/foodcoops/foodcoops.net/milestone/1
It would be great if we can divide the tasks across the team. If you
start with something, self-assign the issue so others know you're
working on it (and unassign if you're moving on to something else). And
if you have an idea, already know how to implement something, or a
relevant question, leave a comment (on any issue). I think we could work
together like this, loosely but with clarity.

- Willem