In #371 we introduced steps to make the scaling up possible even when
the HPA forces a 10% change. The problem is that 10% might not be
sufficient for some specific scaling scenarios.
For example, a an application targeting 12 pods and using a
ScalingSchedule with the value of 10000 to achieve that, will require a
target of 833. With 10 ramp steps the 90% bucket will return a metric of
9000 and the HPA calculates (9000/833) 10.8 pods, rounding to 11 pods.
Once the metric reaches the time to return 100% it will won't be
effective, since the change of the current number of pods (11) and the
desired one (12) is less than 10%.
This commit does not try to tackle this problem completely, since the
10% rule is not fixed, might change among different clusters and is also
dependent on the value given to each ScalingSchedule. Therefore, this
commit makes the number of ramp steps configurable via the
`--scaling-schedule-ramp-steps` config flag, defaulting to 10.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
The HPA has a feature to do not scale up and down when the change in the
metric is less than 10%:
> We'll skip scaling if the ratio is sufficiently close to 1.0 (within a
> globally-configurable tolerance, from the
> `--horizontal-pod-autoscaler-tolerance` flag, which defaults to 0.1.
It could lead to pods scaling up to 10% less than the target for
ScalingSchedules and then not scaling to the actual value if the metric
calculated before was less than 10% of the target.
This commit uses 10 fixed buckets for scaling, this way we know the
metric returned during a scaling event is at least 10% more than a
previous one calculated during the period of ramp up. The same is valid
for the scaling down during a ramp-down
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
this commit updates the k8s dependencies from v0.20.5 to v0.20.9. It
also bundles other dependencies updates.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
This commit add tests to the ZMON package when handling the key used on
the metrics query.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
The `Date` and `Period` fields inside the `Schedule` type of the
`[Cluster]ScalingSchedule` CRDs are optional, but in its current
configuration the generated clients can't update or create these
resources since the empty fields do not pass the validation.
This commit updates the `[Cluster]ScalingSchedule.Schedule[*]` `Date`
and `Period` field to pointers, this way a null value is not validated
and the clients are able to update and create resources. It also updates
the collector code and tests to reflect the change.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
This commit removes the logic that checks for an identifier label in the
http collector config. It also removes the documentation on the README
that mentions that the metric has to be in an old format, removed in
the #219.
Fixes#331
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
During #284 we set the ClusterScalingSchedule scope as `Cluster` but
just for kubebuilder. This flag has no effect in the generated clients.
It means the generated clients expect a namespace argument when handling
ClusterScalingSchedule resources. It works fine to list all of the
resources but makes it impossible to request a specific resource
defining its name.
This commit sets the `+genclient:nonNamespaced` flag for the CRD and
adds the regenerated clients. It does not affect any of the usage of the
clients in this codebase so far.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
This commit adds two new collectors to the adapter:
- ClusterScalingScheduleCollector; and
- ScalingScheduleCollector
Also, it introduces the required collectors plugins, initialization
logic in the server startup, documentation and deployment example
(including the helm chart). A new config flag is created,
`-scaling-schedule`, and allows to enable and to disable the collection
of such metrics. It's disabled by default.
This collectors are the required logic to utilise the CRDs introduced in
the #284 pull request. It makes use of the kubernetes go-client
implementations of a [Store][0] and [Reflector][1].
[0]: https://pkg.go.dev/k8s.io/client-go/tools/cache#Store
[1]: https://pkg.go.dev/k8s.io/client-go/tools/cache#Reflector
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
Testing indicates that metrics API v1beta2 is not a suitable drop-in replacement for v1beta1, and v1 does not exist at this time.
Signed-off-by: Henry Agnew <henryd.agnew@gmail.com>
Some cases and users actually know when their applications will have a
high workload and, therefore, autoscaling support for time based scaling
is a desired feature.
This commit creates the first version of two CRDs called
`ScalingSchedule` and `ClusterScalingSchedule`.
The CRDs describe one or multiples schedules inside them. The schedules
contains the information of when the time based scaling starts, if it
happens once or multiple times, its duration and, a configurable value
that later can be used by HPAs to scale applications.
The only difference between the two CRDs is their scope.
`ClusterScalingSchedule` aims to attend cluster wide schedules, to
multiple applications, while `ScalingSchedule` has to be deployed with
each application into the same namespace.
This commit does not creates any metric, it's a noop change that creates
just the CRD and import tools required to generate the CRD and others
required code, as `deepCopy` functions and clients.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
Use compliant base image according to our tech rules of play:
- KDP-1: use an approved Docker base image and version from the list of
allowed Docker Base Images under the pierone namespace ‘library’.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
NewSession is being deprecated, NewSessionWithConfig handles things
such as role arns. This fixes permissions issues for users using
role arns for service accounts.
Signed-off-by: Ewan Valentine <ewan.valentine89@gmail.com>
* Use labels for mapping metric types to metrics
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
* Log warning when old format is used
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
* Test NewCollector logic for external metrics
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
Allow configuration per metric for connect and request timeouts
when querying pods for JSON metrics.
Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
Fetching metrics from pods sequentially, with a large number of
pods, can result in poor performance when some of those pods have
been terminated by the HPA in a normal scale down event.
Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
Adds a new metric-config option named `rawQuery`. The value of
this option will be appended to the metric `path` as URL query
parameters to be used by the pod's application. E.g.,:
```
metric-config.pods.requests-per-second.json-path/rawQuery: "foo=bar&baz=bop"
```
will apppend `?foo=bar&baz=bop` to the URL.
Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
* This commit adds a --disregard-incompatible-hpas that makes the HPA
provider stop erroring out when a collector cannot be created for a
metric in a HPA. Useful when kube-metrics-adapter runs alongside another
metrics provider. Fixes issue #94.
Signed-off-by: Tomás Pinho <me@tomaspinho.com>
* Make tests pass
Signed-off-by: Tomás Pinho <me@tomaspinho.com>
* Wraps the Plugin Not Found error in a new type that can be checked by the caller of a function to determine if its contents should be logged or added as an event to the HPA, when this HPA is incompatible.
The disregardIncompatibleHPAs is now targetting only the log or addition of the same event.
Signed-off-by: Tomás Pinho <me@tomaspinho.com>
* Invert if expression to select when we should log
CreateNewMetricsCollector errors: don't log when both conditions are true - it's not a PluginNotFoundError
and disregardIncompatibleHPAs flag is set to true. This way, if an error
is NOT PluginNotFoundError it will always be logged, and when it IS
PluginNotFoundError it will only be logged when
disregardIncompatibleHPAs is false.
Signed-off-by: Tomás Pinho <me@tomaspinho.com>
* Remove redundant "whether to"
Signed-off-by: Tomás Pinho <me@tomaspinho.com>
* Add test case for updating HPAs via HPA Provider while disregarding
incompatible HPAs.
Signed-off-by: Tomás Pinho <me@tomaspinho.com>
description:ClusterScalingSchedule describes a cluster scoped time based
metric to be used in autoscaling operations.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScalingScheduleSpec is the spec part of the ScalingSchedule.
properties:
scalingWindowDurationMinutes:
description:Fade the scheduled values in and out over this many minutes.
If unset, the default per-cluster value will be used.
format:int64
type:integer
schedules:
description:Schedules is the list of schedules for this ScalingSchedule
resource. All the schedules defined here will result on the value
to the same metric. New metrics require a new ScalingSchedule resource.
items:
description:Schedule is the schedule details to be used inside
a ScalingSchedule.
properties:
date:
description:Defines the starting date of a OneTime schedule.
It has to be a RFC3339 formated date.
format:date-time
type:string
durationMinutes:
description:The duration in minutes that the configured value
will be returned for the defined schedule.
type:integer
period:
description:Defines the details of a Repeating schedule.
properties:
days:
description:The days that this schedule will be active.
items:
description:ScheduleDay represents the valid inputs for
days in a SchedulePeriod.
enum:
- Sun
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
type:string
type:array
startTime:
description:The startTime has the format HH:MM
pattern:(([0-1][0-9])|([2][0-3])):([0-5][0-9])
type:string
timezone:
description:The location name corresponding to a file in
the IANA Time Zone database, like Europe/Berlin.
type:string
required:
- days
- startTime
- timezone
type:object
type:
description:Defines if the schedule is a OneTime schedule or
Repeating one. If OneTime, date has to be defined. If Repeating,
Period has to be defined.
enum:
- OneTime
- Repeating
type:string
value:
description:The metric value that will be returned for the
description:ClusterScalingSchedule describes a cluster scoped time based
metric to be used in autoscaling operations.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScalingScheduleSpec is the spec part of the ScalingSchedule.
properties:
schedules:
description:Schedules is the list of schedules for this ScalingSchedule
resource. All the schedules defined here will result on the value
to the same metric. New metrics require a new ScalingSchedule resource.
items:
description:Schedule is the schedule details to be used inside
a ScalingSchedule.
properties:
date:
description:Defines the starting date of a OneTime schedule.
It has to be a RFC3339 formated date.
format:date-time
type:string
durationMinutes:
description:The duration in minutes that the configured value
will be returned for the defined schedule.
type:integer
period:
description:Defines the details of a Repeating schedule.
properties:
days:
description:The days that this schedule will be active.
items:
description:ScheduleDay represents the valid inputs for
days in a SchedulePeriod.
enum:
- Sun
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
type:string
type:array
startTime:
description:The startTime has the format HH:MM
pattern:(([0-1][0-9])|([2][0-3])):([0-5][0-9])
type:string
timezone:
description:The location name corresponding to a file in
the IANA Time Zone database, like Europe/Berlin.
type:string
required:
- days
- startTime
- timezone
type:object
type:
description:Defines if the schedule is a OneTime schedule or
Repeating one. If OneTime, date has to be defined. If Repeating,
Period has to be defined.
enum:
- OneTime
- Repeating
type:string
value:
description:The metric value that will be returned for the
description:ScalingSchedule describes a namespaced time based metric to be
used in autoscaling operations.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScalingScheduleSpec is the spec part of the ScalingSchedule.
properties:
schedules:
description:Schedules is the list of schedules for this ScalingSchedule
resource. All the schedules defined here will result on the value
to the same metric. New metrics require a new ScalingSchedule resource.
items:
description:Schedule is the schedule details to be used inside
a ScalingSchedule.
properties:
date:
description:Defines the starting date of a OneTime schedule.
It has to be a RFC3339 formated date.
format:date-time
type:string
durationMinutes:
description:The duration in minutes that the configured value
will be returned for the defined schedule.
type:integer
period:
description:Defines the details of a Repeating schedule.
properties:
days:
description:The days that this schedule will be active.
items:
description:ScheduleDay represents the valid inputs for
days in a SchedulePeriod.
enum:
- Sun
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
type:string
type:array
startTime:
description:The startTime has the format HH:MM
pattern:(([0-1][0-9])|([2][0-3])):([0-5][0-9])
type:string
timezone:
description:The location name corresponding to a file in
the IANA Time Zone database, like Europe/Berlin.
type:string
required:
- days
- startTime
- timezone
type:object
type:
description:Defines if the schedule is a OneTime schedule or
Repeating one. If OneTime, date has to be defined. If Repeating,
Period has to be defined.
enum:
- OneTime
- Repeating
type:string
value:
description:The metric value that will be returned for the
description:ScalingSchedule describes a namespaced time based metric to be
used in autoscaling operations.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScalingScheduleSpec is the spec part of the ScalingSchedule.
properties:
scalingWindowDurationMinutes:
description:Fade the scheduled values in and out over this many minutes.
If unset, the default per-cluster value will be used.
format:int64
type:integer
schedules:
description:Schedules is the list of schedules for this ScalingSchedule
resource. All the schedules defined here will result on the value
to the same metric. New metrics require a new ScalingSchedule resource.
items:
description:Schedule is the schedule details to be used inside
a ScalingSchedule.
properties:
date:
description:Defines the starting date of a OneTime schedule.
It has to be a RFC3339 formated date.
format:date-time
type:string
durationMinutes:
description:The duration in minutes that the configured value
will be returned for the defined schedule.
type:integer
period:
description:Defines the details of a Repeating schedule.
properties:
days:
description:The days that this schedule will be active.
items:
description:ScheduleDay represents the valid inputs for
days in a SchedulePeriod.
enum:
- Sun
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
type:string
type:array
startTime:
description:The startTime has the format HH:MM
pattern:(([0-1][0-9])|([2][0-3])):([0-5][0-9])
type:string
timezone:
description:The location name corresponding to a file in
the IANA Time Zone database, like Europe/Berlin.
type:string
required:
- days
- startTime
- timezone
type:object
type:
description:Defines if the schedule is a OneTime schedule or
Repeating one. If OneTime, date has to be defined. If Repeating,
Period has to be defined.
enum:
- OneTime
- Repeating
type:string
value:
description:The metric value that will be returned for the
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
