In #371 we introduced steps to make the scaling up possible even when
the HPA forces a 10% change. The problem is that 10% might not be
sufficient for some specific scaling scenarios.
For example, a an application targeting 12 pods and using a
ScalingSchedule with the value of 10000 to achieve that, will require a
target of 833. With 10 ramp steps the 90% bucket will return a metric of
9000 and the HPA calculates (9000/833) 10.8 pods, rounding to 11 pods.
Once the metric reaches the time to return 100% it will won't be
effective, since the change of the current number of pods (11) and the
desired one (12) is less than 10%.
This commit does not try to tackle this problem completely, since the
10% rule is not fixed, might change among different clusters and is also
dependent on the value given to each ScalingSchedule. Therefore, this
commit makes the number of ramp steps configurable via the
`--scaling-schedule-ramp-steps` config flag, defaulting to 10.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
The HPA has a feature to do not scale up and down when the change in the
metric is less than 10%:
> We'll skip scaling if the ratio is sufficiently close to 1.0 (within a
> globally-configurable tolerance, from the
> `--horizontal-pod-autoscaler-tolerance` flag, which defaults to 0.1.
It could lead to pods scaling up to 10% less than the target for
ScalingSchedules and then not scaling to the actual value if the metric
calculated before was less than 10% of the target.
This commit uses 10 fixed buckets for scaling, this way we know the
metric returned during a scaling event is at least 10% more than a
previous one calculated during the period of ramp up. The same is valid
for the scaling down during a ramp-down
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
The `Date` and `Period` fields inside the `Schedule` type of the
`[Cluster]ScalingSchedule` CRDs are optional, but in its current
configuration the generated clients can't update or create these
resources since the empty fields do not pass the validation.
This commit updates the `[Cluster]ScalingSchedule.Schedule[*]` `Date`
and `Period` field to pointers, this way a null value is not validated
and the clients are able to update and create resources. It also updates
the collector code and tests to reflect the change.
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
This commit removes the logic that checks for an identifier label in the
http collector config. It also removes the documentation on the README
that mentions that the metric has to be in an old format, removed in
the #219.
Fixes#331
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
This commit adds two new collectors to the adapter:
- ClusterScalingScheduleCollector; and
- ScalingScheduleCollector
Also, it introduces the required collectors plugins, initialization
logic in the server startup, documentation and deployment example
(including the helm chart). A new config flag is created,
`-scaling-schedule`, and allows to enable and to disable the collection
of such metrics. It's disabled by default.
This collectors are the required logic to utilise the CRDs introduced in
the #284 pull request. It makes use of the kubernetes go-client
implementations of a [Store][0] and [Reflector][1].
[0]: https://pkg.go.dev/k8s.io/client-go/tools/cache#Store
[1]: https://pkg.go.dev/k8s.io/client-go/tools/cache#Reflector
Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
* Use labels for mapping metric types to metrics
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
* Log warning when old format is used
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
* Test NewCollector logic for external metrics
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
Allow configuration per metric for connect and request timeouts
when querying pods for JSON metrics.
Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
Fetching metrics from pods sequentially, with a large number of
pods, can result in poor performance when some of those pods have
been terminated by the HPA in a normal scale down event.
Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
Mikkel Oscar Lyderik Larsen