Add some defensive codes for NPE and bump up Kubernetes client version to...

Add some defensive codes for NPE and bump up Kubernetes client version to expose exception stack trace (#6327)

CHANGES.md

@@ -15,6 +15,7 @@ Release Notes.
 #### OAP-Backend
 * Allow user-defined `JAVA_OPTS` in the startup script.
 * Metrics combination API supports abandoning results.
+* Add some defensive codes for NPE and bump up Kubernetes client version to expose exception stack trace.
 
 #### UI
 Update selector scroller to show in all pages.

LICENSE

@@ -224,7 +224,7 @@ The text of each license is the standard Apache 2.0 license.
    flatbuffers files from istio/proxy: https://github.com/istio/proxy Apache 2.0
    mvnw files from https://github.com/takari/maven-wrapper Apache 2.0
    svg files from skywalking-ui/src/assets/icons: https://github.com/google/material-design-icons Apache 2.0
-    
+
 ========================================================================
 BSD licenses
 ========================================================================

dist-material/release-docs/LICENSE

@@ -249,7 +249,7 @@ The text of each license is the standard Apache 2.0 license.
     Eclipse (Jetty) 9.4.28.v20200408: https://www.eclipse.org/jetty/ , Apache 2.0 and Eclipse Public License 1.0
     SnakeYAML 1.18: http://www.snakeyaml.org , Apache 2.0
     Joda-Time 2.10.5: http://www.joda.org/joda-time/ , Apache 2.0
-    Joda-Convert 1.2: http://www.joda.org/joda-convert/ , Apache 2.0
+    Joda-Convert 2.2.1: http://www.joda.org/joda-convert/ , Apache 2.0
     Spring Framework 4.3.14.RELEASE: https://github.com/spring-projects/spring-framework, Apache 2.0
     Spring Boot 1.5.10: https://spring.io/, Apache 2.0
     Spring Cloud Config 1.4.1: https://github.com/spring-cloud/spring-cloud-config, Apache-2.0
@@ -269,8 +269,8 @@ The text of each license is the standard Apache 2.0 license.
     Apache: commons-collections 3.2.2: https://github.com/apache/commons-collections, Apache 2.0
     Apache: commons-configuration 1.8: https://github.com/apache/commons-configuration, Apache 2.0
     Apache: commons-io 2.4: https://github.com/apache/commons-io, Apache 2.0
-    Apache: commons-compress 1.19: https://github.com/apache/commons-compress, Apache 2.0
-    Apache: commons-collections4 4.1: https://mvnrepository.com/artifact/org.apache.commons/commons-collections4, Apache 2.0
+    Apache: commons-compress 1.20: https://github.com/apache/commons-compress, Apache 2.0
+    Apache: commons-collections4 4.4: https://mvnrepository.com/artifact/org.apache.commons/commons-collections4, Apache 2.0
     Apache: tomcat 8.5.27: https://github.com/apache/tomcat/tree/trunk, Apache 2.0
     Apache: freemarker 2.3.28: https://github.com/apache/freemarker, Apache 2.0
     netty 4.1.42: https://github.com/netty/netty/blob/4.1/LICENSE.txt, Apache 2.0
@@ -307,7 +307,7 @@ The text of each license is the standard Apache 2.0 license.
     HikariCP 3.1.0: https://github.com/brettwooldridge/HikariCP, Apache 2.0
     zipkin 2.9.1: https://github.com/openzipkin/zipkin, Apache 2.0
     sharding-jdbc-core 2.0.3: https://github.com/sharding-sphere/sharding-sphere, Apache 2.0
-    kubernetes-client 8.0.0: https://github.com/kubernetes-client/java, Apache 2.0
+    kubernetes-client 10.0.0: https://github.com/kubernetes-client/java, Apache 2.0
     proto files from istio/istio: https://github.com/istio/istio  Apache 2.0
     proto files from istio/api: https://github.com/istio/api      Apache 2.0
     nacos 1.3.1: https://github.com/alibaba/nacos, Apache 2.0
@@ -332,11 +332,11 @@ The text of each license is the standard Apache 2.0 license.
     moshi 1.5.0: https://github.com/square/moshi, Apache 2.0
     logging-interceptor 3.13.1: https://github.com/square/okhttp/tree/master/okhttp-logging-interceptor, Apache 2.0
     msgpack-core 0.8.16: https://github.com/msgpack/msgpack-java, Apache 2.0
-    sundr-codegen 0.21.0: https://mvnrepository.com/artifact/io.sundr/sundr-codegen, Apache 2.0
-    sundr-core 0.21.0: https://mvnrepository.com/artifact/io.sundr/sundr-core, Apache 2.0
-    swagger-annotations 1.5.22: https://mvnrepository.com/artifact/io.swagger.core.v3/swagger-annotations, Apache 2.0
-    resourcecify-annotations 0.21.0: https://mvnrepository.com/artifact/io.sundr/resourcecify-annotations, Apache 2.0
-    jose4j 0.7.0: https://mvnrepository.com/artifact/org.bitbucket.b_c/jose4j, Apache 2.0
+    sundr-codegen 0.22.0: https://mvnrepository.com/artifact/io.sundr/sundr-codegen, Apache 2.0
+    sundr-core 0.22.0: https://mvnrepository.com/artifact/io.sundr/sundr-core, Apache 2.0
+    swagger-annotations 1.6.2: https://mvnrepository.com/artifact/io.swagger.core.v3/swagger-annotations, Apache 2.0
+    resourcecify-annotations 0.22.0: https://mvnrepository.com/artifact/io.sundr/resourcecify-annotations, Apache 2.0
+    jose4j 0.7.2: https://mvnrepository.com/artifact/org.bitbucket.b_c/jose4j, Apache 2.0
     converter-moshi 2.5.0: https://mvnrepository.com/artifact/com.squareup.retrofit2/converter-moshi, Apache 2.0
     vavr 0.10.3: https://github.com/vavr-io/vavr, Apache 2.0
     kafka-clients 2.4.1: https://github.com/apache/kafka, Apache 2.0
@@ -346,6 +346,7 @@ The text of each license is the standard Apache 2.0 license.
     mvel 2.4.8: https://github.com/mvel/mvel, Apache 2.0
     okio 1.13.0: https://github.com/square/okio Apache 2.0
     caffeine 2.6.2: https://github.com/ben-manes/caffeine Apache 2.0
+    simpleclient_httpserver from prometheus https://github.com/prometheus/client_java Apache 2.0
 
 ========================================================================
 MIT licenses
@@ -359,9 +360,9 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
     GraphQL java 8.0: https://github.com/graphql-java/graphql-java , MIT
     GraphQL Java Tools 5.2.3: https://github.com/graphql-java/graphql-java-tools , MIT
     jopt-simple 5.0.2: https://github.com/jopt-simple/jopt-simple , MIT
-    bcpkix-jdk15on 1.61: http://www.bouncycastle.org/licence.html , MIT
-    bcprov-jdk15on 1.61: http://www.bouncycastle.org/licence.html , MIT
-    bcprov-ext-jdk15on 1.61: http://www.bouncycastle.org/licence.html , MIT
+    bcpkix-jdk15on 1.66: http://www.bouncycastle.org/licence.html , MIT
+    bcprov-jdk15on 1.66: http://www.bouncycastle.org/licence.html , MIT
+    bcprov-ext-jdk15on 1.66: http://www.bouncycastle.org/licence.html , MIT
     minimal-json 0.9.5:  https://github.com/ralfstx/minimal-json, MIT
     checker-qual 2.8.1: https://github.com/typetools/checker-framework, MIT
     influxdb-java 2.15: https://github.com/influxdata/influxdb-java, MIT
@@ -449,7 +450,7 @@ popper.js	1.14.7:	https://github.com/FezVrasta/popper.js	MIT
 vue-datepicker-local	1.0.19:	https://github.com/weifeiyue/vue-datepicker-local	MIT
 vue-js-modal	1.3.31:	https://github.com/euvl/vue-js-modal	MIT
 lodash	4.17.15:	https://github.com/lodash/lodash	MIT
-gson-fire	1.8.3:	https://mvnrepository.com/artifact/io.gsonfire/gson-fire	MIT
+gson-fire	1.8.4:	https://mvnrepository.com/artifact/io.gsonfire/gson-fire	MIT
 
 ========================================
 Apache 2.0 licenses

oap-server/pom.xml

@@ -68,7 +68,7 @@
         <commons-io.version>2.6</commons-io.version>
         <elasticsearch.version>6.3.2</elasticsearch.version>
         <joda-time.version>2.10.5</joda-time.version>
-        <kubernetes.version>8.0.0</kubernetes.version>
+        <kubernetes.version>10.0.0</kubernetes.version>
         <hikaricp.version>3.1.0</hikaricp.version>
         <zipkin.version>2.9.1</zipkin.version>
         <caffeine.version>2.6.2</caffeine.version>

oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/k8s/K8SServiceRegistry.java

@@ -37,7 +37,6 @@ import java.io.IOException;
 import java.util.Collections;
 import java.util.List;
 import java.util.Map;
-import java.util.Optional;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.Executors;
@@ -49,7 +48,7 @@ import org.apache.skywalking.oap.server.receiver.envoy.als.ServiceMetaInfo;
 
 import static com.google.common.base.Strings.isNullOrEmpty;
 import static java.util.Objects.isNull;
-import static java.util.Objects.requireNonNull;
+import static java.util.Optional.ofNullable;
 
 @Slf4j
 public class K8SServiceRegistry {
@@ -201,7 +200,7 @@ public class K8SServiceRegistry {
     }
 
     protected void addService(final V1Service service) {
-        Optional.ofNullable(service.getMetadata()).ifPresent(
+        ofNullable(service.getMetadata()).ifPresent(
             metadata -> idServiceMap.put(metadata.getNamespace() + ":" + metadata.getName(), service)
         );
 
@@ -209,13 +208,13 @@ public class K8SServiceRegistry {
     }
 
     protected void removeService(final V1Service service) {
-        Optional.ofNullable(service.getMetadata()).ifPresent(
+        ofNullable(service.getMetadata()).ifPresent(
             metadata -> idServiceMap.remove(metadata.getUid())
         );
     }
 
     protected void addPod(final V1Pod pod) {
-        Optional.ofNullable(pod.getStatus()).ifPresent(
+        ofNullable(pod.getStatus()).ifPresent(
             status -> ipPodMap.put(status.getPodIP(), pod)
         );
 
@@ -223,30 +222,36 @@ public class K8SServiceRegistry {
     }
 
     protected void removePod(final V1Pod pod) {
-        Optional.ofNullable(pod.getStatus()).ifPresent(
+        ofNullable(pod.getStatus()).ifPresent(
             status -> ipPodMap.remove(status.getPodIP())
         );
     }
 
     protected void addEndpoints(final V1Endpoints endpoints) {
-        final String namespace = requireNonNull(endpoints.getMetadata()).getNamespace();
-        final String name = requireNonNull(endpoints.getMetadata()).getName();
+        V1ObjectMeta endpointsMetadata = endpoints.getMetadata();
+        if (isNull(endpointsMetadata)) {
+            log.error("Endpoints metadata is null: {}", endpoints);
+            return;
+        }
+
+        final String namespace = endpointsMetadata.getNamespace();
+        final String name = endpointsMetadata.getName();
 
-        requireNonNull(endpoints.getSubsets()).forEach(
-            subset -> requireNonNull(subset.getAddresses()).forEach(
+        ofNullable(endpoints.getSubsets()).ifPresent(subsets -> subsets.forEach(
+            subset -> ofNullable(subset.getAddresses()).ifPresent(addresses -> addresses.forEach(
                 address -> ipServiceMap.put(address.getIp(), namespace + ":" + name)
-            )
-        );
+            ))
+        ));
 
         recompose();
     }
 
     protected void removeEndpoints(final V1Endpoints endpoints) {
-        requireNonNull(endpoints.getSubsets()).forEach(
-            subset -> requireNonNull(subset.getAddresses()).forEach(
+        ofNullable(endpoints.getSubsets()).ifPresent(subsets -> subsets.forEach(
+            subset -> ofNullable(subset.getAddresses()).ifPresent(addresses -> addresses.forEach(
                 address -> ipServiceMap.remove(address.getIp())
-            )
-        );
+            ))
+        ));
     }
 
     protected List<ServiceMetaInfo.KeyValue> transformLabelsToTags(final Map<String, String> labels) {
@@ -277,7 +282,11 @@ public class K8SServiceRegistry {
             }
 
             final Map<String, Object> context = ImmutableMap.of("service", service, "pod", pod);
-            final V1ObjectMeta podMetadata = requireNonNull(pod.getMetadata());
+            final V1ObjectMeta podMetadata = pod.getMetadata();
+            if (isNull(podMetadata)) {
+                log.warn("Pod metadata is null, {}", pod);
+                return;
+            }
 
             ipServiceMetaInfoMap.computeIfAbsent(ip, unused -> {
                 final ServiceMetaInfo serviceMetaInfo = new ServiceMetaInfo();
@@ -286,7 +295,12 @@ public class K8SServiceRegistry {
                     serviceMetaInfo.setServiceName(serviceNameFormatter.format(context));
                 } catch (Exception e) {
                     log.error("Failed to evaluate service name.", e);
-                    serviceMetaInfo.setServiceName(requireNonNull(service.getMetadata()).getName());
+                    final V1ObjectMeta serviceMetadata = service.getMetadata();
+                    if (isNull(serviceMetadata)) {
+                        log.warn("Service metadata is null, {}", service);
+                        return ServiceMetaInfo.UNKNOWN;
+                    }
+                    serviceMetaInfo.setServiceName(serviceMetadata.getName());
                 }
                 serviceMetaInfo.setServiceInstanceName(
                     String.format("%s.%s", podMetadata.getName(), podMetadata.getNamespace()));

tools/dependencies/known-oap-backend-dependencies-es7.txt

@@ -5,17 +5,17 @@ antlr4-runtime-4.7.1.jar
 aopalliance-1.0.jar
 apollo-client-1.4.0.jar
 apollo-core-1.4.0.jar
-bcpkix-jdk15on-1.61.jar
-bcprov-ext-jdk15on-1.61.jar
-bcprov-jdk15on-1.61.jar
-builder-annotations-0.21.0.jar
+bcpkix-jdk15on-1.66.jar
+bcprov-ext-jdk15on-1.66.jar
+bcprov-jdk15on-1.66.jar
+builder-annotations-0.22.0.jar
 checker-qual-2.8.1.jar
-client-java-8.0.0.jar
-client-java-api-8.0.0.jar
-client-java-proto-8.0.0.jar
+client-java-10.0.0.jar
+client-java-api-10.0.0.jar
+client-java-proto-10.0.0.jar
 commons-codec-1.11.jar
-commons-collections4-4.1.jar
-commons-compress-1.19.jar
+commons-collections4-4.4.jar
+commons-compress-1.20.jar
 commons-dbcp-1.4.jar
 commons-io-2.6.jar
 commons-lang3-3.7.jar
@@ -57,7 +57,7 @@ grpc-protobuf-1.32.1.jar
 grpc-protobuf-lite-1.32.1.jar
 grpc-stub-1.32.1.jar
 gson-2.8.6.jar
-gson-fire-1.8.3.jar
+gson-fire-1.8.4.jar
 guava-28.1-jre.jar
 guice-4.1.0.jar
 h2-1.4.196.jar
@@ -94,10 +94,10 @@ jetty-servlet-9.4.28.v20200408.jar
 jetty-util-9.4.28.v20200408.jar
 jline-0.9.94.jar
 jna-4.5.1.jar
-joda-convert-1.2.jar
+joda-convert-2.2.1.jar
 joda-time-2.10.5.jar
 jopt-simple-4.6.jar
-jose4j-0.7.0.jar
+jose4j-0.7.2.jar
 json-flattener-0.6.0.jar
 jsr305-3.0.2.jar
 kotlin-reflect-1.1.1.jar
@@ -152,16 +152,17 @@ protobuf-java-3.13.0.jar
 protobuf-java-util-3.12.4.jar
 reactive-streams-1.0.2.jar
 reflectasm-1.11.7.jar
-resourcecify-annotations-0.21.0.jar
+resourcecify-annotations-0.22.0.jar
 retrofit-2.3.0.jar
 simpleclient-0.6.0.jar
 simpleclient_common-0.6.0.jar
 simpleclient_hotspot-0.6.0.jar
+simpleclient_httpserver-0.9.0.jar
 slf4j-api-1.7.25.jar
 snakeyaml-1.18.jar
-sundr-codegen-0.21.0.jar
-sundr-core-0.21.0.jar
-swagger-annotations-1.5.22.jar
+sundr-codegen-0.22.0.jar
+sundr-core-0.22.0.jar
+swagger-annotations-1.6.2.jar
 t-digest-3.2.jar
 vavr-0.10.3.jar
 vavr-match-0.10.3.jar

tools/dependencies/known-oap-backend-dependencies.txt

@@ -7,18 +7,18 @@ antlr4-runtime-4.7.1.jar
 aopalliance-1.0.jar
 apollo-client-1.4.0.jar
 apollo-core-1.4.0.jar
-bcpkix-jdk15on-1.61.jar
-bcprov-ext-jdk15on-1.61.jar
-bcprov-jdk15on-1.61.jar
-builder-annotations-0.21.0.jar
+bcpkix-jdk15on-1.66.jar
+bcprov-ext-jdk15on-1.66.jar
+bcprov-jdk15on-1.66.jar
+builder-annotations-0.22.0.jar
 caffeine-2.6.2.jar
 checker-qual-2.8.1.jar
-client-java-8.0.0.jar
-client-java-api-8.0.0.jar
-client-java-proto-8.0.0.jar
+client-java-10.0.0.jar
+client-java-api-10.0.0.jar
+client-java-proto-10.0.0.jar
 commons-codec-1.11.jar
-commons-collections4-4.1.jar
-commons-compress-1.19.jar
+commons-collections4-4.4.jar
+commons-compress-1.20.jar
 commons-dbcp-1.4.jar
 commons-io-2.6.jar
 commons-lang3-3.7.jar
@@ -54,7 +54,7 @@ grpc-protobuf-1.32.1.jar
 grpc-protobuf-lite-1.32.1.jar
 grpc-stub-1.32.1.jar
 gson-2.8.6.jar
-gson-fire-1.8.3.jar
+gson-fire-1.8.4.jar
 guava-28.1-jre.jar
 guice-4.1.0.jar
 h2-1.4.196.jar
@@ -89,10 +89,10 @@ jetty-servlet-9.4.28.v20200408.jar
 jetty-util-9.4.28.v20200408.jar
 jline-0.9.94.jar
 jna-4.5.1.jar
-joda-convert-1.2.jar
+joda-convert-2.2.1.jar
 joda-time-2.10.5.jar
 jopt-simple-4.6.jar
-jose4j-0.7.0.jar
+jose4j-0.7.2.jar
 json-flattener-0.6.0.jar
 jsr305-3.0.2.jar
 kotlin-reflect-1.1.1.jar
@@ -149,16 +149,17 @@ protobuf-java-util-3.12.4.jar
 rank-eval-client-6.3.2.jar
 reactive-streams-1.0.2.jar
 reflectasm-1.11.7.jar
-resourcecify-annotations-0.21.0.jar
+resourcecify-annotations-0.22.0.jar
 retrofit-2.3.0.jar
 simpleclient-0.6.0.jar
 simpleclient_common-0.6.0.jar
 simpleclient_hotspot-0.6.0.jar
+simpleclient_httpserver-0.9.0.jar
 slf4j-api-1.7.25.jar
 snakeyaml-1.18.jar
-sundr-codegen-0.21.0.jar
-sundr-core-0.21.0.jar
-swagger-annotations-1.5.22.jar
+sundr-codegen-0.22.0.jar
+sundr-core-0.22.0.jar
+swagger-annotations-1.6.2.jar
 t-digest-3.2.jar
 vavr-0.10.3.jar
 vavr-match-0.10.3.jar