Compare commits

..
31 Commits
Author SHA1 Message Date
Mikkel Oscar Lyderik Larsen c614b57a63 Serve OpenAPI spec
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
2020-06-26 19:19:15 +02:00
Mikkel Oscar Lyderik Larsen 5bd527b698 Update client-go
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
2020-06-26 18:22:38 +02:00
Mikkel Oscar Lyderik LarsenandGitHub c3a03fd758 Merge pull request #157 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.31.7
Bump github.com/aws/aws-sdk-go from 1.30.24 to 1.31.7
2020-06-02 09:43:49 +02:00
Mikkel Oscar Lyderik LarsenandGitHub 05704c0a6b Merge pull request #158 from zalando-incubator/dependabot/go_modules/github.com/stretchr/testify-1.6.0
Bump github.com/stretchr/testify from 1.5.1 to 1.6.0
2020-06-02 09:43:27 +02:00
Mikkel Oscar Lyderik LarsenandGitHub f3a608fcf7 Merge pull request #159 from zalando-incubator/dependabot/go_modules/github.com/prometheus/common-0.10.0
Bump github.com/prometheus/common from 0.9.1 to 0.10.0
2020-06-02 09:43:03 +02:00
dependabot-preview[bot]andGitHub 223ec9fd89 Bump github.com/prometheus/common from 0.9.1 to 0.10.0
Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.9.1 to 0.10.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](https://github.com/prometheus/common/compare/v0.9.1...v0.10.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-01 10:47:03 +00:00
dependabot-preview[bot]andGitHub 81255aa956 Bump github.com/stretchr/testify from 1.5.1 to 1.6.0
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.5.1 to 1.6.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.5.1...v1.6.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-01 10:46:34 +00:00
dependabot-preview[bot]andGitHub dac44965e3 Bump github.com/aws/aws-sdk-go from 1.30.24 to 1.31.7
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.24 to 1.31.7.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.24...v1.31.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-01 10:46:23 +00:00
Mikkel Oscar Lyderik LarsenandGitHub dbed1570ba Merge pull request #156 from zalando-incubator/networking.k8s.io
Add support for Ingress of API Group networking.k8s.io
2020-05-26 15:09:18 +02:00
Mikkel Oscar Lyderik Larsen ef24244074 Add support for networking.k8s.io
Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
2020-05-26 14:47:46 +02:00
ArjunandGitHub cadf2dff3c Merge pull request #153 from doyshinda/custom_timeouts
Add support for custom timeouts
2020-05-18 10:27:17 +02:00
ArjunandGitHub db83268343 Merge pull request #150 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.30.24
Bump github.com/aws/aws-sdk-go from 1.30.19 to 1.30.24
2020-05-18 10:25:50 +02:00
Mikkel Oscar Lyderik LarsenandGitHub 78c2ef742b Merge pull request #152 from doyshinda/parallel_get_metrics
Fetch pod metrics in parallel
2020-05-18 09:26:14 +02:00
Abe Friesen f0b817629c Add support for custom timeouts
Allow configuration per metric for connect and request timeouts
when querying pods for JSON metrics.

Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
2020-05-16 07:15:58 -06:00
Abe Friesen 3e7b66070c Fetch pod metrics in parallel
Fetching metrics from pods sequentially, with a large number of
pods, can result in poor performance when some of those pods have
been terminated by the HPA in a normal scale down event.

Signed-off-by: Abe Friesen <2319792+doyshinda@users.noreply.github.com>
2020-05-16 06:47:13 -06:00
dependabot-preview[bot]andGitHub f3d33b6132 Bump github.com/aws/aws-sdk-go from 1.30.19 to 1.30.24
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.19 to 1.30.24.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.19...v1.30.24)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-11 10:18:26 +00:00
ArjunandGitHub 7b7af889fb Merge pull request #148 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.30.19
Bump github.com/aws/aws-sdk-go from 1.30.14 to 1.30.19
2020-05-06 12:45:54 +02:00
ArjunandGitHub 2bcaa80ce0 Merge pull request #147 from zalando-incubator/dependabot/go_modules/github.com/prometheus/client_golang-1.6.0
Bump github.com/prometheus/client_golang from 1.5.1 to 1.6.0
2020-05-06 12:45:46 +02:00
ArjunandGitHub a2e6980b25 Merge pull request #146 from zalando-incubator/dependabot/go_modules/github.com/sirupsen/logrus-1.6.0
Bump github.com/sirupsen/logrus from 1.5.0 to 1.6.0
2020-05-06 12:45:23 +02:00
dependabot-preview[bot]andGitHub f0783dcdd8 Bump github.com/aws/aws-sdk-go from 1.30.14 to 1.30.19
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.14 to 1.30.19.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.14...v1.30.19)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-04 09:59:27 +00:00
dependabot-preview[bot]andGitHub 560fb40dae Bump github.com/prometheus/client_golang from 1.5.1 to 1.6.0
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.5.1 to 1.6.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.5.1...v1.6.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-04 09:58:05 +00:00
dependabot-preview[bot]andGitHub 41229e370d Bump github.com/sirupsen/logrus from 1.5.0 to 1.6.0
Bumps [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sirupsen/logrus/compare/v1.5.0...v1.6.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-04 09:57:46 +00:00
ArjunandGitHub 338393c11e Merge pull request #145 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.30.14
Bump github.com/aws/aws-sdk-go from 1.30.9 to 1.30.14
2020-05-02 21:28:27 +02:00
dependabot-preview[bot]andGitHub 8e7d1bc0d6 Bump github.com/aws/aws-sdk-go from 1.30.9 to 1.30.14
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.9 to 1.30.14.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.9...v1.30.14)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-27 10:18:59 +00:00
ArjunandGitHub dc7fb5875b Merge pull request #143 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.30.9
Bump github.com/aws/aws-sdk-go from 1.30.7 to 1.30.9
2020-04-21 06:20:06 +02:00
dependabot-preview[bot]andGitHub 8f70d3493e Bump github.com/aws/aws-sdk-go from 1.30.7 to 1.30.9
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.7 to 1.30.9.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.7...v1.30.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-20 10:00:12 +00:00
Gábor LiptákandGitHub 97ec13d010 Update Travis to Go 1.14 (#140)
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2020-04-12 22:54:49 +02:00
Martin LinkhorstandGitHub 5e20dcd961 Merge pull request #139 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.30.7
Bump github.com/aws/aws-sdk-go from 1.30.3 to 1.30.7
2020-04-09 15:52:18 +02:00
dependabot-preview[bot]andGitHub f27dbc3939 Bump github.com/aws/aws-sdk-go from 1.30.3 to 1.30.7
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.3 to 1.30.7.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.3...v1.30.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-09 08:48:10 +00:00
Mikkel Oscar Lyderik LarsenandGitHub 6443613930 Merge pull request #135 from zalando-incubator/dependabot/go_modules/github.com/aws/aws-sdk-go-1.30.3
Bump github.com/aws/aws-sdk-go from 1.30.2 to 1.30.3
2020-04-03 12:35:00 +02:00
dependabot-preview[bot]andGitHub d9c2e50f2c Bump github.com/aws/aws-sdk-go from 1.30.2 to 1.30.3
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.2 to 1.30.3.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.2...v1.30.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-03 08:48:22 +00:00
19 changed files with 14245 additions and 125 deletions
+1 -1
View File
@@ -2,7 +2,7 @@ language: go
dist: xenial
go:
- "1.13.x"
- "1.14.x"
env:
- GO111MODULE=on GOLANGCI_RELEASE="v1.21.0"
+22 -12
View File
@@ -1,19 +1,22 @@
.PHONY: clean test check build.local build.linux build.osx build.docker build.push
BINARY ?= kube-metrics-adapter
VERSION ?= $(shell git describe --tags --always --dirty)
IMAGE ?= registry-write.opensource.zalan.do/teapot/$(BINARY)
TAG ?= $(VERSION)
SOURCES = $(shell find . -name '*.go')
DOCKERFILE ?= Dockerfile
GOPKGS = $(shell go list ./...)
BUILD_FLAGS ?= -v
LDFLAGS ?= -X main.version=$(VERSION) -w -s
BINARY ?= kube-metrics-adapter
VERSION ?= $(shell git describe --tags --always --dirty)
IMAGE ?= registry-write.opensource.zalan.do/teapot/$(BINARY)
TAG ?= $(VERSION)
SOURCES = $(shell find . -name '*.go')
DOCKERFILE ?= Dockerfile
GOPKGS = $(shell go list ./...)
GO_OPENAPI_GEN = ./build/openapi-gen
OPENAPI_GEN = pkg/apiserver/generated/openapi/zz_generated.openapi.go
BUILD_FLAGS ?= -v
LDFLAGS ?= -X main.version=$(VERSION) -w -s
default: build.local
clean:
rm -rf build
rm -rf $(OPENAPI_GEN)
test:
go test -v $(GOPKGS)
@@ -22,17 +25,24 @@ check:
go mod download
golangci-lint run --timeout=2m ./...
$(GO_OPENAPI_GEN):
mkdir -p build
GOBIN=$(shell pwd)/build go install k8s.io/kube-openapi/cmd/openapi-gen
$(OPENAPI_GEN): $(GO_OPENAPI_GEN)
$(GO_OPENAPI_GEN) -o . --go-header-file hack/boilerplate.go.txt --logtostderr -i k8s.io/metrics/pkg/apis/custom_metrics,k8s.io/metrics/pkg/apis/custom_metrics/v1beta1,k8s.io/metrics/pkg/apis/custom_metrics/v1beta2,k8s.io/metrics/pkg/apis/external_metrics,k8s.io/metrics/pkg/apis/external_metrics/v1beta1,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/apimachinery/pkg/api/resource,k8s.io/apimachinery/pkg/version,k8s.io/api/core/v1 -p pkg/apiserver/generated/openapi -O zz_generated.openapi -r /dev/null
build.local: build/$(BINARY)
build.linux: build/linux/$(BINARY)
build.osx: build/osx/$(BINARY)
build/$(BINARY): go.mod $(SOURCES)
build/$(BINARY): go.mod $(OPENAPI_GEN) $(SOURCES)
CGO_ENABLED=0 go build -o build/$(BINARY) $(BUILD_FLAGS) -ldflags "$(LDFLAGS)" .
build/linux/$(BINARY): go.mod $(SOURCES)
build/linux/$(BINARY): go.mod $(OPENAPI_GEN) $(SOURCES)
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build $(BUILD_FLAGS) -o build/linux/$(BINARY) -ldflags "$(LDFLAGS)" .
build/osx/$(BINARY): go.mod $(SOURCES)
build/osx/$(BINARY): go.mod $(OPENAPI_GEN) $(SOURCES)
GOOS=darwin GOARCH=amd64 CGO_ENABLED=0 go build $(BUILD_FLAGS) -o build/osx/$(BINARY) -ldflags "$(LDFLAGS)" .
build.docker: build.linux
+8
View File
@@ -166,6 +166,14 @@ will create a URL like this:
http://<podIP>:9090/metrics?foo=bar&baz=bop
```
There are also configuration options for custom (connect and request) timeouts when querying pods for metrics:
```yaml
metric-config.pods.requests-per-second.json-path/request-timeout: 2s
metric-config.pods.requests-per-second.json-path/connect-timeout: 500ms
```
The default for both of the above values is 15 seconds.
## Prometheus collector
The Prometheus collector is a generic collector which can map Prometheus
+16 -13
View File
@@ -2,31 +2,34 @@ module github.com/zalando-incubator/kube-metrics-adapter
require (
github.com/NYTimes/gziphandler v1.0.1 // indirect
github.com/aws/aws-sdk-go v1.30.2
github.com/aws/aws-sdk-go v1.31.7
github.com/go-openapi/spec v0.19.3
github.com/googleapis/gnostic v0.2.0 // indirect
github.com/influxdata/influxdb-client-go v0.1.5
github.com/kubernetes-incubator/custom-metrics-apiserver v0.0.0-20200323093244-5046ce1afe6b
github.com/kubernetes-incubator/custom-metrics-apiserver v0.0.0-20200618121405-54026617ec44
github.com/lib/pq v1.2.0 // indirect
github.com/mattn/go-colorable v0.1.4 // indirect
github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852
github.com/onsi/ginkgo v1.11.0 // indirect
github.com/onsi/gomega v1.8.1 // indirect
github.com/prometheus/client_golang v1.5.1
github.com/prometheus/common v0.9.1
github.com/sirupsen/logrus v1.5.0
github.com/prometheus/client_golang v1.6.0
github.com/prometheus/common v0.10.0
github.com/sirupsen/logrus v1.6.0
github.com/spf13/cobra v0.0.7
github.com/stretchr/testify v1.5.1
github.com/stretchr/testify v1.6.0
github.com/zalando-incubator/cluster-lifecycle-manager v0.0.0-20180921141935-824b77fb1f84
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
golang.org/x/tools v0.0.0-20200204192400-7124308813f3 // indirect
gonum.org/v1/netlib v0.0.0-20190331212654-76723241ea4e // indirect
honnef.co/go/tools v0.0.1-2020.1.3 // indirect
k8s.io/api v0.17.3
k8s.io/apimachinery v0.17.4
k8s.io/client-go v0.17.3
k8s.io/component-base v0.17.3
k8s.io/api v0.18.4
k8s.io/apiextensions-apiserver v0.18.4
k8s.io/apimachinery v0.18.4
k8s.io/apiserver v0.18.4
k8s.io/client-go v0.18.4
k8s.io/component-base v0.18.4
k8s.io/klog v1.0.0
k8s.io/metrics v0.17.3
k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6
k8s.io/metrics v0.18.4
)
go 1.13
+168 -49
View File
File diff suppressed because it is too large Load Diff
+15
View File
@@ -0,0 +1,15 @@
/*
Copyright YEAR The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
File diff suppressed because it is too large Load Diff
+10 -3
View File
@@ -26,21 +26,28 @@ func NewJSONPathMetricsGetter(httpClient *http.Client, aggregatorFunc Aggregator
return &JSONPathMetricsGetter{client: httpClient, aggregator: aggregatorFunc, jsonPath: compiledPath}
}
func DefaultMetricsHTTPClient() *http.Client {
var DefaultRequestTimeout = 15 * time.Second
var DefaultConnectTimeout = 15 * time.Second
func CustomMetricsHTTPClient(requestTimeout time.Duration, connectTimeout time.Duration) *http.Client {
client := &http.Client{
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: 15 * time.Second,
Timeout: connectTimeout,
}).DialContext,
MaxIdleConns: 50,
IdleConnTimeout: 90 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
},
Timeout: 15 * time.Second,
Timeout: requestTimeout,
}
return client
}
func DefaultMetricsHTTPClient() *http.Client {
return CustomMetricsHTTPClient(DefaultRequestTimeout, DefaultConnectTimeout)
}
// GetMetric gets metric from pod by fetching json metrics from the pods metric
// endpoint and extracting the desired value using the specified json path
// query.
+28 -1
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"net/url"
"strconv"
"time"
"github.com/oliveagle/jsonpath"
v1 "k8s.io/api/core/v1"
@@ -72,7 +73,33 @@ func NewPodMetricsJSONPathGetter(config map[string]string) (*PodMetricsJSONPathG
return nil, err
}
}
getter.metricGetter = NewJSONPathMetricsGetter(DefaultMetricsHTTPClient(), aggregator, jsonPath)
requestTimeout := DefaultRequestTimeout
connectTimeout := DefaultConnectTimeout
if v, ok := config["request-timeout"]; ok {
d, err := time.ParseDuration(v)
if err != nil {
return nil, err
}
if d < 0 {
return nil, fmt.Errorf("Invalid request-timeout config value: %s", v)
}
requestTimeout = d
}
if v, ok := config["connect-timeout"]; ok {
d, err := time.ParseDuration(v)
if err != nil {
return nil, err
}
if d < 0 {
return nil, fmt.Errorf("Invalid connect-timeout config value: %s", v)
}
connectTimeout = d
}
getter.metricGetter = NewJSONPathMetricsGetter(CustomMetricsHTTPClient(requestTimeout, connectTimeout), aggregator, jsonPath)
return &getter, nil
}
@@ -3,6 +3,7 @@ package httpmetrics
import (
"fmt"
"testing"
"time"
"github.com/oliveagle/jsonpath"
"github.com/stretchr/testify/require"
@@ -131,3 +132,67 @@ func TestBuildMetricsURL(t *testing.T) {
receivedURLNoQuery := getterWithNoQuery.buildMetricsURL(ip)
require.Equal(t, receivedURLNoQuery.String(), expectedURLNoQuery)
}
func TestCustomTimeouts(t *testing.T) {
scheme := "http"
port := "9090"
path := "/v1/test/"
// Test no custom options results in default timeouts
defaultConfig := map[string]string{
"json-key": "$.value",
"scheme": scheme,
"path": path,
"port": port,
}
defaultTime := time.Duration(15000) * time.Millisecond
defaultGetter, err1 := NewPodMetricsJSONPathGetter(defaultConfig)
require.NoError(t, err1)
require.Equal(t, defaultGetter.metricGetter.client.Timeout, defaultTime)
// Test with custom request timeout
configWithRequestTimeout := map[string]string{
"json-key": "$.value",
"scheme": scheme,
"path": path,
"port": port,
"request-timeout": "978ms",
}
exectedTimeout := time.Duration(978) * time.Millisecond
customRequestGetter, err2 := NewPodMetricsJSONPathGetter(configWithRequestTimeout)
require.NoError(t, err2)
require.Equal(t, customRequestGetter.metricGetter.client.Timeout, exectedTimeout)
// Test with custom connect timeout. Unfortunately, it seems there's no way to access the
// connect timeout of the client struct to actually verify it's set :/
configWithConnectTimeout := map[string]string{
"json-key": "$.value",
"scheme": scheme,
"path": path,
"port": port,
"connect-timeout": "512ms",
}
customRequestGetter, err3 := NewPodMetricsJSONPathGetter(configWithConnectTimeout)
require.NoError(t, err3)
configWithInvalidTimeout := map[string]string{
"json-key": "$.value",
"scheme": scheme,
"path": path,
"port": port,
"request-timeout": "-256ms",
}
_, err4 := NewPodMetricsJSONPathGetter(configWithInvalidTimeout)
require.Error(t, err4)
configWithInvalidTimeout = map[string]string{
"json-key": "$.value",
"scheme": scheme,
"path": path,
"port": port,
"connect-timeout": "-256ms",
}
_, err5 := NewPodMetricsJSONPathGetter(configWithInvalidTimeout)
require.Error(t, err5)
}
+39 -26
View File
@@ -1,6 +1,7 @@
package collector
import (
"context"
"fmt"
"net/http"
"time"
@@ -8,6 +9,7 @@ import (
log "github.com/sirupsen/logrus"
"github.com/zalando-incubator/kube-metrics-adapter/pkg/collector/httpmetrics"
autoscalingv2 "k8s.io/api/autoscaling/v2beta2"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
@@ -80,37 +82,25 @@ func (c *PodCollector) GetMetrics() ([]CollectedMetric, error) {
LabelSelector: labels.Set(c.podLabelSelector.MatchLabels).String(),
}
pods, err := c.client.CoreV1().Pods(c.namespace).List(opts)
pods, err := c.client.CoreV1().Pods(c.namespace).List(context.TODO(), opts)
if err != nil {
return nil, err
}
values := make([]CollectedMetric, 0, len(pods.Items))
// TODO: get metrics in parallel
ch := make(chan CollectedMetric)
errCh := make(chan error)
for _, pod := range pods.Items {
value, err := c.Getter.GetMetric(&pod)
if err != nil {
c.logger.Errorf("Failed to get metrics from pod '%s/%s': %v", pod.Namespace, pod.Name, err)
continue
}
go c.getPodMetric(pod, ch, errCh)
}
metricValue := CollectedMetric{
Type: c.metricType,
Custom: custom_metrics.MetricValue{
DescribedObject: custom_metrics.ObjectReference{
APIVersion: "v1",
Kind: "Pod",
Name: pod.Name,
Namespace: pod.Namespace,
},
Metric: custom_metrics.MetricIdentifier{Name: c.metric.Name, Selector: c.podLabelSelector},
Timestamp: metav1.Time{Time: time.Now().UTC()},
Value: *resource.NewMilliQuantity(int64(value*1000), resource.DecimalSI),
},
values := make([]CollectedMetric, 0, len(pods.Items))
for i := 0; i < len(pods.Items); i++ {
select {
case err := <-errCh:
c.logger.Error(err)
case resp := <-ch:
values = append(values, resp)
}
values = append(values, metricValue)
}
return values, nil
@@ -120,16 +110,39 @@ func (c *PodCollector) Interval() time.Duration {
return c.interval
}
func (c *PodCollector) getPodMetric(pod corev1.Pod, ch chan CollectedMetric, errCh chan error) {
value, err := c.Getter.GetMetric(&pod)
if err != nil {
errCh <- fmt.Errorf("Failed to get metrics from pod '%s/%s': %v", pod.Namespace, pod.Name, err)
return
}
ch <- CollectedMetric{
Type: c.metricType,
Custom: custom_metrics.MetricValue{
DescribedObject: custom_metrics.ObjectReference{
APIVersion: "v1",
Kind: "Pod",
Name: pod.Name,
Namespace: pod.Namespace,
},
Metric: custom_metrics.MetricIdentifier{Name: c.metric.Name, Selector: c.podLabelSelector},
Timestamp: metav1.Time{Time: time.Now().UTC()},
Value: *resource.NewMilliQuantity(int64(value*1000), resource.DecimalSI),
},
}
}
func getPodLabelSelector(client kubernetes.Interface, hpa *autoscalingv2.HorizontalPodAutoscaler) (*metav1.LabelSelector, error) {
switch hpa.Spec.ScaleTargetRef.Kind {
case "Deployment":
deployment, err := client.AppsV1().Deployments(hpa.Namespace).Get(hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
deployment, err := client.AppsV1().Deployments(hpa.Namespace).Get(context.TODO(), hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
return deployment.Spec.Selector, nil
case "StatefulSet":
sts, err := client.AppsV1().StatefulSets(hpa.Namespace).Get(hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
sts, err := client.AppsV1().StatefulSets(hpa.Namespace).Get(context.TODO(), hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
+10 -4
View File
@@ -1,11 +1,13 @@
package collector
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"sync"
"testing"
"time"
@@ -55,7 +57,7 @@ func TestPodCollector(t *testing.T) {
for _, m := range metrics {
values = append(values, m.Custom.Value.Value())
}
require.Equal(t, tc.result, values)
require.ElementsMatch(t, tc.result, values)
})
}
}
@@ -68,9 +70,13 @@ type testMetricsHandler struct {
calledCounter uint
t *testing.T
metricsPath string
sync.RWMutex
}
func (h *testMetricsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
h.Lock()
defer h.Unlock()
require.Equal(h.t, h.metricsPath, r.URL.Path)
require.Less(h.t, int(h.calledCounter), len(h.values))
response, err := json.Marshal(testMetricResponse{Values: h.values[h.calledCounter]})
@@ -110,7 +116,7 @@ func makeTestPods(t *testing.T, testServer string, metricName string, port strin
PodIP: testServer,
},
}
_, err := client.CoreV1().Pods(testNamespace).Create(testPod)
_, err := client.CoreV1().Pods(testNamespace).Create(context.TODO(), testPod, v1.CreateOptions{})
require.NoError(t, err)
}
}
@@ -124,7 +130,7 @@ func makeTestDeployment(t *testing.T, client kubernetes.Interface) *appsv1.Deplo
},
},
}
_, err := client.AppsV1().Deployments(testNamespace).Create(&deployment)
_, err := client.AppsV1().Deployments(testNamespace).Create(context.TODO(), &deployment, v1.CreateOptions{})
require.NoError(t, err)
return &deployment
@@ -144,7 +150,7 @@ func makeTestHPA(t *testing.T, client kubernetes.Interface) *autoscalingv2.Horiz
},
},
}
_, err := client.AutoscalingV2beta2().HorizontalPodAutoscalers("test-namespace").Create(hpa)
_, err := client.AutoscalingV2beta2().HorizontalPodAutoscalers("test-namespace").Create(context.TODO(), hpa, v1.CreateOptions{})
require.NoError(t, err)
return hpa
}
+4 -3
View File
@@ -1,6 +1,7 @@
package collector
import (
"context"
"encoding/json"
"errors"
"fmt"
@@ -125,7 +126,7 @@ func getWeights(ingressAnnotations map[string]string, backendAnnotations []strin
// getCollector returns a collector for getting the metrics.
func (c *SkipperCollector) getCollector() (Collector, error) {
ingress, err := c.client.ExtensionsV1beta1().Ingresses(c.objectReference.Namespace).Get(c.objectReference.Name, metav1.GetOptions{})
ingress, err := c.client.NetworkingV1beta1().Ingresses(c.objectReference.Namespace).Get(context.TODO(), c.objectReference.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
@@ -207,13 +208,13 @@ func targetRefReplicas(client kubernetes.Interface, hpa *autoscalingv2.Horizonta
var replicas int32
switch hpa.Spec.ScaleTargetRef.Kind {
case "Deployment":
deployment, err := client.AppsV1().Deployments(hpa.Namespace).Get(hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
deployment, err := client.AppsV1().Deployments(hpa.Namespace).Get(context.TODO(), hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
if err != nil {
return 0, err
}
replicas = deployment.Status.Replicas
case "StatefulSet":
sts, err := client.AppsV1().StatefulSets(hpa.Namespace).Get(hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
sts, err := client.AppsV1().StatefulSets(hpa.Namespace).Get(context.TODO(), hpa.Spec.ScaleTargetRef.Name, metav1.GetOptions{})
if err != nil {
return 0, err
}
+9 -8
View File
@@ -1,6 +1,7 @@
package collector
import (
"context"
"encoding/json"
"fmt"
"testing"
@@ -10,7 +11,7 @@ import (
appsv1 "k8s.io/api/apps/v1"
autoscalingv2 "k8s.io/api/autoscaling/v2beta2"
corev1 "k8s.io/api/core/v1"
"k8s.io/api/extensions/v1beta1"
v1beta1 "k8s.io/api/networking/v1beta1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
@@ -32,7 +33,7 @@ func TestTargetRefReplicasDeployments(t *testing.T) {
// Create an HPA with the deployment as ref
hpa, err := client.AutoscalingV2beta2().HorizontalPodAutoscalers(deployment.Namespace).
Create(newHPA(defaultNamespace, name, "Deployment"))
Create(context.TODO(), newHPA(defaultNamespace, name, "Deployment"), metav1.CreateOptions{})
require.NoError(t, err)
replicas, err := targetRefReplicas(client, hpa)
@@ -49,7 +50,7 @@ func TestTargetRefReplicasStatefulSets(t *testing.T) {
// Create an HPA with the statefulSet as ref
hpa, err := client.AutoscalingV2beta2().HorizontalPodAutoscalers(statefulSet.Namespace).
Create(newHPA(defaultNamespace, name, "StatefulSet"))
Create(context.TODO(), newHPA(defaultNamespace, name, "StatefulSet"), metav1.CreateOptions{})
require.NoError(t, err)
replicas, err := targetRefReplicas(client, hpa)
@@ -73,7 +74,7 @@ func newHPA(namesapce string, refName string, refKind string) *autoscalingv2.Hor
}
func newDeployment(client *fake.Clientset, namespace string, name string, replicas, readyReplicas int32) (*appsv1.Deployment, error) {
return client.AppsV1().Deployments(namespace).Create(&appsv1.Deployment{
return client.AppsV1().Deployments(namespace).Create(context.TODO(), &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: name,
Namespace: namespace,
@@ -83,11 +84,11 @@ func newDeployment(client *fake.Clientset, namespace string, name string, replic
ReadyReplicas: replicas,
Replicas: readyReplicas,
},
})
}, metav1.CreateOptions{})
}
func newStatefulSet(client *fake.Clientset, namespace string, name string) (*appsv1.StatefulSet, error) {
return client.AppsV1().StatefulSets(namespace).Create(&appsv1.StatefulSet{
return client.AppsV1().StatefulSets(namespace).Create(context.TODO(), &appsv1.StatefulSet{
ObjectMeta: metav1.ObjectMeta{
Name: name,
Namespace: namespace,
@@ -96,7 +97,7 @@ func newStatefulSet(client *fake.Clientset, namespace string, name string) (*app
ReadyReplicas: 1,
Replicas: 2,
},
})
}, metav1.CreateOptions{})
}
func TestSkipperCollector(t *testing.T) {
@@ -382,7 +383,7 @@ func makeIngress(client kubernetes.Interface, namespace, ingressName, backend st
Host: hostname,
})
}
_, err := client.ExtensionsV1beta1().Ingresses(namespace).Create(ingress)
_, err := client.NetworkingV1beta1().Ingresses(namespace).Create(context.TODO(), ingress, metav1.CreateOptions{})
return err
}
+1 -1
View File
@@ -119,7 +119,7 @@ func (p *HPAProvider) Run(ctx context.Context) {
func (p *HPAProvider) updateHPAs() error {
p.logger.Info("Looking for HPAs")
hpas, err := p.client.AutoscalingV2beta2().HorizontalPodAutoscalers(metav1.NamespaceAll).List(metav1.ListOptions{})
hpas, err := p.client.AutoscalingV2beta2().HorizontalPodAutoscalers(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
if err != nil {
return err
}
+3 -3
View File
@@ -70,7 +70,7 @@ func TestUpdateHPAs(t *testing.T) {
fakeClient := fake.NewSimpleClientset()
var err error
hpa, err = fakeClient.AutoscalingV2beta2().HorizontalPodAutoscalers("default").Create(hpa)
hpa, err = fakeClient.AutoscalingV2beta2().HorizontalPodAutoscalers("default").Create(context.TODO(), hpa, metav1.CreateOptions{})
require.NoError(t, err)
collectorFactory := collector.NewCollectorFactory()
@@ -86,7 +86,7 @@ func TestUpdateHPAs(t *testing.T) {
// update HPA
hpa.Annotations["metric-config.pods.requests-per-second.json-path/port"] = "8080"
_, err = fakeClient.AutoscalingV2beta2().HorizontalPodAutoscalers("default").Update(hpa)
_, err = fakeClient.AutoscalingV2beta2().HorizontalPodAutoscalers("default").Update(context.TODO(), hpa, metav1.UpdateOptions{})
require.NoError(t, err)
err = provider.updateHPAs()
@@ -134,7 +134,7 @@ func TestUpdateHPAsDisregardingIncompatibleHPA(t *testing.T) {
fakeClient := fake.NewSimpleClientset()
var err error
_, err = fakeClient.AutoscalingV2beta2().HorizontalPodAutoscalers("default").Create(hpa)
_, err = fakeClient.AutoscalingV2beta2().HorizontalPodAutoscalers("default").Create(context.TODO(), hpa, metav1.CreateOptions{})
require.NoError(t, err)
collectorFactory := collector.NewCollectorFactory()
+7 -1
View File
@@ -69,9 +69,15 @@ func (s *MetricStore) insertCustomMetric(value custom_metrics.MetricValue) {
Resource: "pods",
}
case "Ingress":
// group can be either `extentions` or `networking.k8s.io`
group := "extensions"
gv, err := schema.ParseGroupVersion(value.DescribedObject.APIVersion)
if err == nil {
group = gv.Group
}
groupResource = schema.GroupResource{
Resource: "ingresses",
Group: "extensions",
Group: group,
}
}
+8
View File
@@ -29,10 +29,14 @@ import (
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/spf13/cobra"
"github.com/zalando-incubator/cluster-lifecycle-manager/pkg/credentials-loader/platformiam"
generatedopenapi "github.com/zalando-incubator/kube-metrics-adapter/pkg/apiserver/generated/openapi"
"github.com/zalando-incubator/kube-metrics-adapter/pkg/collector"
"github.com/zalando-incubator/kube-metrics-adapter/pkg/provider"
"github.com/zalando-incubator/kube-metrics-adapter/pkg/zmon"
"golang.org/x/oauth2"
"k8s.io/apiextensions-apiserver/pkg/apiserver"
openapinamer "k8s.io/apiserver/pkg/endpoints/openapi"
genericapiserver "k8s.io/apiserver/pkg/server"
"k8s.io/client-go/informers"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
@@ -126,6 +130,10 @@ func (o AdapterServerOptions) RunCustomMetricsAdapterServer(stopCh <-chan struct
return err
}
config.GenericConfig.OpenAPIConfig = genericapiserver.DefaultOpenAPIConfig(generatedopenapi.GetOpenAPIDefinitions, openapinamer.NewDefinitionNamer(apiserver.Scheme))
config.GenericConfig.OpenAPIConfig.Info.Title = "kube-metrics-adapter"
config.GenericConfig.OpenAPIConfig.Info.Version = "1.0.0"
var clientConfig *rest.Config
if len(o.RemoteKubeConfigFile) > 0 {
loadingRules := &clientcmd.ClientConfigLoadingRules{ExplicitPath: o.RemoteKubeConfigFile}
+7
View File
@@ -0,0 +1,7 @@
// +build tools
package main
import (
_ "k8s.io/kube-openapi/cmd/openapi-gen"
)