lunny/helm-chart
1
0
Fork 0
Code Issues 36 Pull Requests 14 Actions Packages Projects Releases 93 Wiki Activity

Multiple replicas error when cron.GIT_GC_REPOS is defined #488

New Issue
Closed
opened 2023-08-22 14:16:02 +00:00 by ovisan · 14 comments
ovisan commented 2023-08-22 14:16:02 +00:00 (Migrated from gitea.com)
Copy Link

Hi,
Tried to use replicaCount: 3 in the helm chart, but i am getting the following error:

Helm upgrade failed: template: gitea/templates/gitea/deployment.yaml:29:28: executing "gitea/templates/gitea/deployment.yaml" at <include │
│ (print $.Template.BasePath "/gitea/config.yaml") .>: error calling include: template: gitea/templates/gitea/config.yaml:28:16: executing "gitea/templates/gitea/config.yaml" at <.Values.gitea.config │
│ .cron.GIT_GC_REPOS>: nil pointer evaluating interface {}.GIT_GC_REPOS

That was the only change in the helm values. Worked fine with replicaCount: 1.

Any clue?
Thanks.

Hi, Tried to use replicaCount: 3 in the helm chart, but i am getting the following error: Helm upgrade failed: template: gitea/templates/gitea/deployment.yaml:29:28: executing "gitea/templates/gitea/deployment.yaml" at <include │ │ (print $.Template.BasePath "/gitea/config.yaml") .>: error calling include: template: gitea/templates/gitea/config.yaml:28:16: executing "gitea/templates/gitea/config.yaml" at <.Values.gitea.config │ │ .cron.GIT_GC_REPOS>: nil pointer evaluating interface {}.GIT_GC_REPOS That was the only change in the helm values. Worked fine with replicaCount: 1. Any clue? Thanks.
pat-s commented 2023-08-22 17:37:42 +00:00 (Migrated from gitea.com)
Copy Link

In 35fcb41ce2/templates/gitea/config.yaml (L28-L30) we check whether the value exists and require it be false.

Do you have something set for this value? I don't really see why the template would error but I guess your answer might reveal possible required actions.

For a short-term fix you can delete the setting from your app.ini.

In https://gitea.com/gitea/helm-chart/src/commit/35fcb41ce2d03b44186cc82d4ea619dc2fcb6f7d/templates/gitea/config.yaml#L28-L30 we check whether the value exists and require it be `false`. Do you have something set for this value? I don't really see why the template would error but I guess your answer might reveal possible required actions. For a short-term fix you can delete the setting from your `app.ini`.
ovisan commented 2023-08-23 08:01:39 +00:00 (Migrated from gitea.com)
Copy Link

@pat-s i have no setting defined for cron, this is the config section i use:

      config:
        APP_NAME: "Gitea"
        repository:
          ROOT: "~/gitea-repositories"
        database:
          DB_TYPE: postgres
          SCHEMA: public
          SSL_MODE: require
        server:
          ROOT_URL: "https://<REDACTED>"
          DISABLE_SSH: true
          OFFLINE_MODE: true
        service:
          DEFAULT_ALLOW_CREATE_ORGANIZATION: false
          DEFAULT_USER_IS_RESTRICTED: true
          DEFAULT_KEEP_EMAIL_PRIVATE: true
          DEFAULT_USER_VISIBILITY: "private"
          DEFAULT_ORG_VISIBILITY: "private"
          DISABLE_REGISTRATION: true
          REQUIRE_SIGNIN_VIEW: true
          explore:
            REQUIRE_SIGNIN_VIEW: true
            DISABLE_USERS_PAGE: true
        openid:
          ENABLE_OPENID_SIGNIN: true
          ENABLE_OPENID_SIGNUP: true
        oauth2_client:
          ENABLE_AUTO_REGISTRATION: true
        ui:
          SHOW_USER_EMAIL: false
        admin:
          DISABLE_REGULAR_ORG_CREATION: true
      oauth:
        - name: "Keycloak"
          provider: "openidConnect"
          existingSecret: "gitea-oauth"
          autoDiscoverUrl: "https://<REDACTEDE>/auth/realms/<REDACTED>/.well-known/openid-configuration"
@pat-s i have no setting defined for cron, this is the config section i use: ``` config: APP_NAME: "Gitea" repository: ROOT: "~/gitea-repositories" database: DB_TYPE: postgres SCHEMA: public SSL_MODE: require server: ROOT_URL: "https://<REDACTED>" DISABLE_SSH: true OFFLINE_MODE: true service: DEFAULT_ALLOW_CREATE_ORGANIZATION: false DEFAULT_USER_IS_RESTRICTED: true DEFAULT_KEEP_EMAIL_PRIVATE: true DEFAULT_USER_VISIBILITY: "private" DEFAULT_ORG_VISIBILITY: "private" DISABLE_REGISTRATION: true REQUIRE_SIGNIN_VIEW: true explore: REQUIRE_SIGNIN_VIEW: true DISABLE_USERS_PAGE: true openid: ENABLE_OPENID_SIGNIN: true ENABLE_OPENID_SIGNUP: true oauth2_client: ENABLE_AUTO_REGISTRATION: true ui: SHOW_USER_EMAIL: false admin: DISABLE_REGULAR_ORG_CREATION: true oauth: - name: "Keycloak" provider: "openidConnect" existingSecret: "gitea-oauth" autoDiscoverUrl: "https://<REDACTEDE>/auth/realms/<REDACTED>/.well-known/openid-configuration"
pat-s commented 2023-08-26 11:23:33 +00:00 (Migrated from gitea.com)
Copy Link

I see the following getting auto-added to app.ini

[cron]
UPDATE_CHECKER = map[ENABLED:false]
DELETE_MISSING_REPOS = map[ENABLED:true]
GIT_GC_REPOS = map[ENABLED:true]
RESYNC_ALL_SSHKEYS = map[ENABLED:true RUN_AT_START:true]
RESYNC_ALL_HOOKS = map[ENABLED:true]
DELETE_OLD_SYSTEM_NOTICES = map[ENABLED:true]

I can replicate your issue. I think it's because GIT_GC_REPOS is a map and the template is checking for a single value definition. Will check for a fix. Thanks for reporting!

I see the following getting auto-added to `app.ini` ``` [cron] UPDATE_CHECKER = map[ENABLED:false] DELETE_MISSING_REPOS = map[ENABLED:true] GIT_GC_REPOS = map[ENABLED:true] RESYNC_ALL_SSHKEYS = map[ENABLED:true RUN_AT_START:true] RESYNC_ALL_HOOKS = map[ENABLED:true] DELETE_OLD_SYSTEM_NOTICES = map[ENABLED:true] ``` I can replicate your issue. I think it's because `GIT_GC_REPOS` is a map and the template is checking for a single value definition. Will check for a fix. Thanks for reporting!
👍 1
ovisan commented 2023-08-28 15:48:30 +00:00 (Migrated from gitea.com)
Copy Link

@pat-s Tried to upgrade today, the only changed values:
are chart version to 9.2.1 and image to 1.20.3 and replicaCount: 3

Warning  error   2m5s (x8 over 3m43s)  helm-controller  Helm upgrade failed: template: gitea/templates/gitea/deployment.yaml:29:28: executing "gitea/templates/gitea/deployment.yaml" at <include (pr │
│ int $.Template.BasePath "/gitea/config.yaml") .>: error calling include: template: gitea/templates/gitea/config.yaml:28:40: executing "gitea/templates/gitea/config.yaml" at <"cron.GIT_GC_REPOS">: wro │
│ ng type for value; expected map[string]interface {}; got string
@pat-s Tried to upgrade today, the only changed values: are chart version to 9.2.1 and image to 1.20.3 and replicaCount: 3 ``` Warning error 2m5s (x8 over 3m43s) helm-controller Helm upgrade failed: template: gitea/templates/gitea/deployment.yaml:29:28: executing "gitea/templates/gitea/deployment.yaml" at <include (pr │ │ int $.Template.BasePath "/gitea/config.yaml") .>: error calling include: template: gitea/templates/gitea/config.yaml:28:40: executing "gitea/templates/gitea/config.yaml" at <"cron.GIT_GC_REPOS">: wro │ │ ng type for value; expected map[string]interface {}; got string
pat-s commented 2023-08-28 16:50:43 +00:00 (Migrated from gitea.com)
Copy Link

Please delete any existing cron entries in app.ini - there might be some due to the previous upscaling attempts which do not get removed automatically.

Please delete any existing `cron` entries in `app.ini` - there might be some due to the previous upscaling attempts which do not get removed automatically.
ovisan commented 2023-08-28 17:12:41 +00:00 (Migrated from gitea.com)
Copy Link

@pat-s the container got recreated automatically when updating, there was no persistent data other than the data stored in postgresql cluster

@pat-s the container got recreated automatically when updating, there was no persistent data other than the data stored in postgresql cluster
justusbunsi commented 2023-08-28 17:22:31 +00:00 (Migrated from gitea.com)
Copy Link

@ovisan

@pat-s the container got recreated automatically when updating, there was no persistent data other than the data stored in postgresql cluster

Careful with this. The app.ini must be persistent. Otherwise, some data won't be accessible anymore. Some settings in app.ini must stay the same once created.

What confuses me is the syntax that is generated. Isn't this the syntax for cron-related configurations? 4803766f7a/custom/conf/app.example.ini (L2175) (similar for other settings). Have I missed a major change in the app.ini design?

@ovisan >@pat-s the container got recreated automatically when updating, there was no persistent data other than the data stored in postgresql cluster Careful with this. The app.ini must be persistent. Otherwise, some data won't be accessible anymore. Some settings in app.ini must stay the same once created. What confuses me is the syntax that is generated. Isn't this the syntax for cron-related configurations? https://github.com/go-gitea/gitea/blob/4803766f7a5d1ca86a1e62949dcd9a9ff8d0f64c/custom/conf/app.example.ini#L2175 (similar for other settings). Have I missed a major change in the app.ini design?
ovisan commented 2023-08-28 18:09:41 +00:00 (Migrated from gitea.com)
Copy Link

@justusbunsi There is nothing cron related in the app.ini, but by having more than one replicaCount it will trigger the error from above. The app.ini is not persisted because it lives in the gitea container and it is generated each time the helm chart is being deployed.

Contents of app.ini
gitea-57b4c7fbb7-4n5vh:/# cat /data/gitea/conf/app.ini

APP_NAME = Gitea
WORK_PATH = /data

[indexer]
ISSUE_INDEXER_TYPE = db

[server]
LFS_JWT_SECRET = <REDACTED>
ROOT_URL = https://<REDACTED>
OFFLINE_MODE = true
ENABLE_PPROF = false
SSH_PORT = 22
APP_DATA_PATH = /data
SSH_LISTEN_PORT = 2222
PROTOCOL = http
HTTP_PORT = 3000
START_SSH_SERVER = true
DISABLE_SSH = true
SSH_DOMAIN = <REDACTED>
DOMAIN = <REDACTED>

[database]
NAME = gitea
SSL_MODE = require
DB_TYPE = postgres
HOST = gitea-postgres-primary.<REDACTED>.svc
USER = gitea
PASSWD = <REDACTED>
SCHEMA = public

[service]
DEFAULT_KEEP_EMAIL_PRIVATE = true
REQUIRE_SIGNIN_VIEW = true
DISABLE_REGISTRATION = true
DEFAULT_USER_IS_RESTRICTED = true
EXPLORE = map[DISABLE_USERS_PAGE:true REQUIRE_SIGNIN_VIEW:true]
DEFAULT_ORG_VISIBILITY = private
DEFAULT_USER_VISIBILITY = private
DEFAULT_ALLOW_CREATE_ORGANIZATION = false

[session]
PROVIDER_CONFIG = redis+cluster://:@gitea-redis-cluster-headless.<REDACTED>.svc.cluster.local:6379/0?pool_size=100&idle_timeout=180s&
PROVIDER = redis-cluster
SESSION_LIFE_TIME = 1800
GC_INTERVAL_TIME = 1800
COOKIE_SECURE = true

[repository]
ROOT = ~/gitea-repositories

[cache]
HOST = redis+cluster://:@gitea-redis-cluster-headless.<REDACTED>.svc.cluster.local:6379/0?pool_size=100&idle_timeout=180s&
ENABLED = true
ADAPTER = redis

[security]
INTERNAL_TOKEN = <REDACTED>
SECRET_KEY = <REDACTED>
INSTALL_LOCK = true

[openid]
ENABLE_OPENID_SIGNIN = true
ENABLE_OPENID_SIGNUP = true

[queue]
TYPE = redis
CONN_STR = redis+cluster://:@gitea-redis-cluster-headless.<REDACTED>.svc.cluster.local:6379/0?pool_size=100&idle_timeout=180s&

[admin]
DISABLE_REGULAR_ORG_CREATION = true

[ui]
SHOW_USER_EMAIL = false

[metrics]
ENABLED = true

[oauth2_client]
ENABLE_AUTO_REGISTRATION = true

[oauth2]
JWT_SECRET = <REDACTED>
@justusbunsi There is nothing cron related in the app.ini, but by having more than one replicaCount it will trigger the error from above. The app.ini is not persisted because it lives in the gitea container and it is generated each time the helm chart is being deployed. Contents of app.ini `gitea-57b4c7fbb7-4n5vh:/# cat /data/gitea/conf/app.ini` ``` APP_NAME = Gitea WORK_PATH = /data [indexer] ISSUE_INDEXER_TYPE = db [server] LFS_JWT_SECRET = <REDACTED> ROOT_URL = https://<REDACTED> OFFLINE_MODE = true ENABLE_PPROF = false SSH_PORT = 22 APP_DATA_PATH = /data SSH_LISTEN_PORT = 2222 PROTOCOL = http HTTP_PORT = 3000 START_SSH_SERVER = true DISABLE_SSH = true SSH_DOMAIN = <REDACTED> DOMAIN = <REDACTED> [database] NAME = gitea SSL_MODE = require DB_TYPE = postgres HOST = gitea-postgres-primary.<REDACTED>.svc USER = gitea PASSWD = <REDACTED> SCHEMA = public [service] DEFAULT_KEEP_EMAIL_PRIVATE = true REQUIRE_SIGNIN_VIEW = true DISABLE_REGISTRATION = true DEFAULT_USER_IS_RESTRICTED = true EXPLORE = map[DISABLE_USERS_PAGE:true REQUIRE_SIGNIN_VIEW:true] DEFAULT_ORG_VISIBILITY = private DEFAULT_USER_VISIBILITY = private DEFAULT_ALLOW_CREATE_ORGANIZATION = false [session] PROVIDER_CONFIG = redis+cluster://:@gitea-redis-cluster-headless.<REDACTED>.svc.cluster.local:6379/0?pool_size=100&idle_timeout=180s& PROVIDER = redis-cluster SESSION_LIFE_TIME = 1800 GC_INTERVAL_TIME = 1800 COOKIE_SECURE = true [repository] ROOT = ~/gitea-repositories [cache] HOST = redis+cluster://:@gitea-redis-cluster-headless.<REDACTED>.svc.cluster.local:6379/0?pool_size=100&idle_timeout=180s& ENABLED = true ADAPTER = redis [security] INTERNAL_TOKEN = <REDACTED> SECRET_KEY = <REDACTED> INSTALL_LOCK = true [openid] ENABLE_OPENID_SIGNIN = true ENABLE_OPENID_SIGNUP = true [queue] TYPE = redis CONN_STR = redis+cluster://:@gitea-redis-cluster-headless.<REDACTED>.svc.cluster.local:6379/0?pool_size=100&idle_timeout=180s& [admin] DISABLE_REGULAR_ORG_CREATION = true [ui] SHOW_USER_EMAIL = false [metrics] ENABLED = true [oauth2_client] ENABLE_AUTO_REGISTRATION = true [oauth2] JWT_SECRET = <REDACTED>
justusbunsi commented 2023-08-28 19:26:27 +00:00 (Migrated from gitea.com)
Copy Link

I think I got confused by the comment from pat-s. (https://gitea.com/gitea/helm-chart/issues/488#issuecomment-747888) 🙂

I think I got confused by the comment from pat-s. (https://gitea.com/gitea/helm-chart/issues/488#issuecomment-747888) 🙂
pat-s commented 2023-08-30 07:24:20 +00:00 (Migrated from gitea.com)
Copy Link

Careful with this. The app.ini must be persistent. Otherwise, some data won't be accessible anymore. Some settings in app.ini must stay the same once created.

I am aware and know which settings are persistet atm. This is also the tricky part in solving this issue in general 🙂 I haven't got enough time to wrap my head around a good solution yet.

Have I missed a major change in the app.ini design?

I was also unsure myself but it seems that most config sections now use the "X.Y" notation as the section name. This is somewhat an issue with how helm treats a "X.Y" notation, i.e. it usually nests these values unless they are quoted or treated with the get() function in helm templates. Which is what initially caused the error reported here.

> Careful with this. The app.ini must be persistent. Otherwise, some data won't be accessible anymore. Some settings in app.ini must stay the same once created. I am aware and know which settings are persistet atm. This is also the tricky part in solving this issue in general 🙂 I haven't got enough time to wrap my head around a good solution yet. > Have I missed a major change in the app.ini design? I was also unsure myself but it seems that most config sections now use the "X.Y" notation as the section name. This is somewhat an issue with how helm treats a "X.Y" notation, i.e. it usually nests these values unless they are quoted or treated with the `get()` function in helm templates. Which is what initially caused the error reported here.
ovisan commented 2023-08-30 12:15:49 +00:00 (Migrated from gitea.com)
Copy Link

@pat-s @justusbunsi should this be reopened or moved into a separate bug?

@pat-s @justusbunsi should this be reopened or moved into a separate bug?
ovisan commented 2023-08-30 12:19:02 +00:00 (Migrated from gitea.com)
Copy Link

@pat-s @justusbunsi https://github.com/go-gitea/gitea/issues/26821#issuecomment-1699020365 maybe related

@pat-s @justusbunsi https://github.com/go-gitea/gitea/issues/26821#issuecomment-1699020365 maybe related
pat-s commented 2023-08-31 08:27:34 +00:00 (Migrated from gitea.com)
Copy Link

I wouldn't say this is an issue of the helm chart but in the end how env-to-ini parses helm config values. I am also not sure if this is an issue or desired behavior and actually a user error.
It would be nice if env-to-ini would be able to account for this.

Example:

The following values notation

CRON:
  UPDATE_CHECKER:
    ENABLED: true

is translated into

[cron]
UPDATE_CHECKER = map[ENABLED:false]

instead of

[cron.update_checker]
ENABLED=true

because env-to-ini doesn't convert nested sections into the section header but into a subsequent map.

On the user end, the following works

"cron.update_checker":
  ENABLED: true
I wouldn't say this is an issue of the helm chart but in the end how `env-to-ini` parses helm config values. I am also not sure if this is an issue or desired behavior and actually a user error. It would be nice if `env-to-ini` would be able to account for this. Example: The following values notation ``` CRON: UPDATE_CHECKER: ENABLED: true ``` is translated into ``` [cron] UPDATE_CHECKER = map[ENABLED:false] ``` instead of ``` [cron.update_checker] ENABLED=true ``` because `env-to-ini` doesn't convert nested sections into the section header but into a subsequent `map`. On the user end, the following works ``` "cron.update_checker": ENABLED: true ```
pat-s commented 2023-08-31 08:48:15 +00:00 (Migrated from gitea.com)
Copy Link

https://github.com/go-gitea/gitea/issues/26838

https://github.com/go-gitea/gitea/issues/26838
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
has/backport
in progress
invalid
kind/breaking
kind/bug
kind/build
kind/dependency
kind/deployment
kind/docs
kind/enhancement
kind/feature
kind/lint
kind/proposal
kind/question
kind/refactor
kind/security
kind/testing
kind/translation
kind/ui
need/backport
priority/critical
priority/low
priority/maybe
priority/medium
reviewed/duplicate
reviewed/invalid
reviewed/wontfix
skip-changelog
status/blocked
status/needs-feedback
status/needs-reviews
status/wip
upstream/gitea
upstream/other
No Label
kind/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: lunny/helm-chart#488
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?