Publish chart on Dockerhub #525

Merged
pat-s merged 11 commits from refs/pull/525/head into main 2023-11-14 23:27:28 +00:00
pat-s commented 2023-10-05 19:49:28 +00:00 (Migrated from gitea.com)

As title.

As title.
pat-s commented 2023-10-05 19:59:51 +00:00 (Migrated from gitea.com)

blocked by #526

blocked by #526
techknowlogick (Migrated from gitea.com) reviewed 2023-10-11 08:19:27 +00:00
justusbunsi (Migrated from gitea.com) reviewed 2023-10-11 08:19:28 +00:00
justusbunsi (Migrated from gitea.com) reviewed 2023-10-11 18:16:24 +00:00
@ -92,2 +82,3 @@
Gitea can be run with an external database and cache.
This chart provides those dependencies, which can be enabled, or disabled via configuration.
### HA Dependencies
justusbunsi (Migrated from gitea.com) commented 2023-10-11 18:16:24 +00:00

For the "old" way you'd still need to run helm repo update.

For the "old" way you'd still need to run `helm repo update`.
justusbunsi (Migrated from gitea.com) commented 2023-10-11 18:35:25 +00:00

OCI registries cannot be added as Helm repositories, afaik. Installing a chart from an OCI registry is a oneliner: helm install gitea oci://registry-1.docker.io/gitea/helm with an optional --version "<chart-version>".

OCI registries cannot be added as Helm repositories, afaik. Installing a chart from an OCI registry is a oneliner: `helm install gitea oci://registry-1.docker.io/gitea/helm` with an optional `--version "<chart-version>"`.
justusbunsi (Migrated from gitea.com) commented 2023-10-12 08:23:09 +00:00

Ah interesting. TF didn't complain but I guess because it directly uses the helm install approach. Thanks!

Ah interesting. TF didn't complain but I guess because it directly uses the `helm install` approach. Thanks!
justusbunsi (Migrated from gitea.com) reviewed 2023-10-13 09:16:54 +00:00
justusbunsi (Migrated from gitea.com) reviewed 2023-10-13 12:18:54 +00:00
justusbunsi (Migrated from gitea.com) left a comment

Uhm, @pat-s. I've played with the helm push commands towards DockerHub. Looks like it wouldn't work how we intend to split Gitea images from Helm Chart. Please give me a few minutes to write a more in-depth experience with screenshots and what I understood so far.

Uhm, @pat-s. I've played with the `helm push` commands towards DockerHub. Looks like it wouldn't work how we intend to split Gitea images from Helm Chart. Please give me a few minutes to write a more in-depth experience with screenshots and what I understood so far.
justusbunsi commented 2023-10-13 13:03:46 +00:00 (Migrated from gitea.com)

Why I rejected the current state

This is based on the official docs for how to push OCI artifacts to DockerHub and testing with my own account and a private repository.

What is a DockerHub repository

Looking at the docker/demo repository, you can have different artifact types. But every tag must be unique. If not, pushing another artifact type over an existing tag, the previously existing tag still exists but is only accessible through the digest. The tag is rewritten. I've tested this in my private repository justusbunsi/helm.

Having an image tag 0.1.0

image

After pushing a Helm Chart with version 0.1.0 into this repository

image

What happens when accessing the image via tag

image

What would work

image

Pushing a Helm Chart to DockerHub

When pushing a Helm Chart to DockerHub, the OCI address is oci://registry-1.docker.io/<username>. Trying to use an address like oci://registry-1.docker.io/<username>/<repository> results in an error message like Error: push access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed.
There are other OCI-compatible registries like Harbor where you could do that. Even with more subpaths. But DockerHub seem to not support that.

The helm command automatically determines what artifact it is processing by reading the Chart.yaml in the tgz file. In our case, the artifact name would be gitea. The version would be 9.6.0 and newer versions when starting this. helm push will combine the given OCI address and the artifact name and would push an artifact with tag 9.6.0 to oci://registry-1.docker.io/gitea/gitea, leading to a mixed repository like the docker/demo is. This may be no issue right now due to Gitea not bumping the official major version and the Helm Chart already being on major 9. But at some point this can result in issues like above in the first section.

EDIT: When using helm pull oci://registry-1.docker.io/<username>/<repository> without an explicit version via --version flag, Helm extracts all tags in this repository, sorts them and compares them according to semver. Tags like 1.20-nightly would be 1.20 for Helm. This can create lots of headache.

The gitea/helm repository

Having experienced the above, I would say that the gitea/helm repository won't be used as long as we don't name our Helm Chart helm and publish versions for that. There is no way to overrule the Chart name during helm package command, like for appVersion or version.

A possible solution?

When looking at Bitnami Helm Charts, their helm install instructions refer to oci://registry-1.docker.io/bitnamicharts/<chart>. It seems Bitnami uses a bitnami user for providing Docker images and a bitnamicharts user for providing Helm Charts. That's why they can name their repositories identical for both artifacts, without colliding tags.


Right now, I am really unsure what to do. The most clean solution would be using different DockerHub accounts. But that would double payments, right? Am I missing anything that would allow us to make our current approach work?

# Why I rejected the current state This is based on the official docs for how to push [OCI artifacts to DockerHub](https://docs.docker.com/docker-hub/oci-artifacts/#push-a-helm-chart) and testing with my own account and a private repository. ## What is a DockerHub repository Looking at the [docker/demo](https://hub.docker.com/r/docker/demo/tags) repository, you can have different artifact types. But every tag must be unique. If not, pushing another artifact type over an existing tag, the previously existing tag still exists but is only accessible through the digest. The tag is rewritten. I've tested this in my private repository `justusbunsi/helm`. <details> <summary>Having an image tag 0.1.0</summary> ![image](/attachments/e7daca87-9a61-44e3-a45c-74010982bb5c) </details> <details> <summary>After pushing a Helm Chart with version 0.1.0 into this repository</summary> ![image](/attachments/1ef18af8-77ed-4da3-be8b-fcc0588cf98e) <details> <summary>What happens when accessing the image via tag</summary> ![image](/attachments/6a238592-d68a-4155-ba8a-bbe816e7c5ac) </details> <details> <summary>What would work</summary> ![image](/attachments/1805bc04-5bd9-4f19-b053-9c08d9ff2565) </details> </details> ## Pushing a Helm Chart to DockerHub When pushing a Helm Chart to DockerHub, the OCI address is `oci://registry-1.docker.io/<username>`. Trying to use an address like `oci://registry-1.docker.io/<username>/<repository>` results in an error message like `Error: push access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed`. There are other OCI-compatible registries like Harbor where you could do that. Even with more subpaths. But DockerHub seem to not support that. The helm command automatically determines what artifact it is processing by reading the Chart.yaml in the tgz file. In our case, the artifact name would be `gitea`. The version would be `9.6.0` and newer versions when starting this. `helm push` will combine the given OCI address and the artifact name and would push an artifact with tag `9.6.0` to `oci://registry-1.docker.io/gitea/gitea`, leading to a mixed repository like the `docker/demo` is. This may be no issue right now due to Gitea not bumping the official major version and the Helm Chart already being on major 9. But at some point this can result in issues like above in the first section. EDIT: When using `helm pull oci://registry-1.docker.io/<username>/<repository>` without an explicit version via `--version` flag, Helm extracts all tags in this repository, sorts them and compares them according to semver. Tags like `1.20-nightly` would be `1.20` for Helm. This can create lots of headache. ## The [gitea/helm](https://hub.docker.com/r/gitea/helm) repository Having experienced the above, I would say that the `gitea/helm` repository won't be used as long as we don't name our Helm Chart `helm` and publish versions for that. There is no way to overrule the Chart name during [helm package](https://helm.sh/docs/helm/helm_package/) command, like for appVersion or version. ## A possible solution? When looking at Bitnami Helm Charts, their `helm install` instructions refer to `oci://registry-1.docker.io/bitnamicharts/<chart>`. It seems Bitnami uses a [`bitnami` user](https://hub.docker.com/u/bitnami) for providing Docker images and a [`bitnamicharts` user](https://hub.docker.com/u/bitnamicharts) for providing Helm Charts. That's why they can name their repositories identical for both artifacts, without colliding tags. --- Right now, I am really unsure what to do. The most clean solution would be using different DockerHub accounts. But that would double payments, right? Am I missing _anything_ that would allow us to make our current approach work?
pat-s commented 2023-10-13 13:50:14 +00:00 (Migrated from gitea.com)

Oh wow, thanks for the deep dive! Which was truly needed given the points you discovered (and I was not aware about and learned today). Thanks a lot!

I agree that under these circumstances we should not release it in it's current form as this would result in a mixed gitea/gitea repository.

And yes, renaming the chart is also not an option imo only for this feature.

Right now, I am really unsure what to do. The most clean solution would be using different DockerHub accounts. But that would double payments, right? Am I missing anything that would allow us to make our current approach work?

You don't need to pay necessarily for a "working" dockerhub account - and if this one is only used for the charts, I don't think that going with the free version would be an issue.
So if @techknowlogick would agree, I think a second account for publishing the charts would be a possible solution.
It could be named giteacharts or similar and then the final URL would be oci://registry-1.docker.io/giteacharts/gitea.

Oh wow, thanks for the deep dive! Which was truly needed given the points you discovered (and I was not aware about and learned today). Thanks a lot! I agree that under these circumstances we should not release it in it's current form as this would result in a mixed `gitea/gitea` repository. And yes, renaming the chart is also not an option imo only for this feature. > Right now, I am really unsure what to do. The most clean solution would be using different DockerHub accounts. But that would double payments, right? Am I missing anything that would allow us to make our current approach work? You don't need to pay necessarily for a "working" dockerhub account - and if this one is only used for the charts, I don't think that going with the free version would be an issue. So if @techknowlogick would agree, I think a second account for publishing the charts *would be a possible solution*. It could be named `giteacharts` or similar and then the final URL would be `oci://registry-1.docker.io/giteacharts/gitea`.
lunny commented 2023-10-31 12:28:00 +00:00 (Migrated from gitea.com)

I have created the new org giteacharts please try again. secrets names has been changed.

I have created the new org `giteacharts` please try again. secrets names has been changed.
pat-s commented 2023-11-06 08:09:26 +00:00 (Migrated from gitea.com)

@justusbunsi After lunny created https://hub.docker.com/r/giteacharts/gitea, I've adjusted the secrets and README. I think this approach should be "safe" now?

@justusbunsi After lunny created https://hub.docker.com/r/giteacharts/gitea, I've adjusted the secrets and README. I think this approach should be "safe" now?
justusbunsi (Migrated from gitea.com) reviewed 2023-11-06 19:12:37 +00:00
@ -56,3 +54,3 @@
# push to dockerhub
echo ${{ secrets.DOCKER_CHARTS_PASSWORD }} | helm registry login -u ${{ secrets.DOCKER_CHARTS_USERNAME }} registry-1.docker.io --password-stdin
helm push gitea/gitea-${GITHUB_REF#refs/tags/v}.tgz oci://registry-1.docker.io/giteacharts
helm push gitea-${GITHUB_REF#refs/tags/v}.tgz oci://registry-1.docker.io/giteacharts
justusbunsi (Migrated from gitea.com) commented 2023-11-06 19:12:37 +00:00

This should push the chart to oci://registry-1.docker.io/giteacharts. See section "Pushing a Helm Chart to DockerHub" in my deep dive.

This should push the chart to `oci://registry-1.docker.io/giteacharts`. See section "Pushing a Helm Chart to DockerHub" in my deep dive.
justusbunsi (Migrated from gitea.com) commented 2023-11-06 20:02:47 +00:00

Well spotted 👍

Well spotted 👍
justusbunsi (Migrated from gitea.com) approved these changes 2023-11-11 19:11:14 +00:00
justusbunsi (Migrated from gitea.com) left a comment

Should be good to go now. You mentioned that DockerHub charts will are available "since v9.6.0": Do we wait with merging the PR until we release that version?

Should be good to go now. You mentioned that DockerHub charts will are available "since v9.6.0": Do we wait with merging the PR until we release that version?
Sign in to join this conversation.
No description provided.