Application won't start with opentracing.spring.cloud.reactor.enabled=true

[kkroner8451]

We have an application that is using Spring Cloud Gateway and is connected to Redis and Jaeger that is deployed in Pivotal Cloud Foundry.

Problem

When not run in Cloud Foundry (i.e. on a host machine) and still connected to Redis/Jaeger everything seems to load fine with the Tracer automatically created using the spring.application.name property which is set in bootstrap.yml file.
bootstrap.yml

opentracing:
  jaeger:
    udp-sender:
      host: jaeger.tld
      port: ###
    tags:
      environment: ci
      datacenter: dc1
    remote-controlled-sampler:
      host: jaeger.tld
      port: ###
    enabled: true
  spring:
    web:
      enabled: true

pom.xml

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.3.7.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
    <java.version>11</java.version>
    <spring-cloud-services.version>2.3.0.RELEASE</spring-cloud-services.version>
    <spring-cloud.version>Hoxton.SR9</spring-cloud.version>
</properties>

<dependencies>
  <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-gateway</artifactId>
  </dependency>
  <dependency>
      <groupId>io.opentracing.contrib</groupId>
      <artifactId>opentracing-spring-jaeger-cloud-starter</artifactId>
      <version>3.2.2</version>
  </dependency>
  <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-redis</artifactId>
  </dependency>
  
  <dependency>
      <groupId>org.springframework.session</groupId>
      <artifactId>spring-session-data-redis</artifactId>
  </dependency>
</dependencies>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.pivotal.spring.cloud</groupId>
            <artifactId>spring-cloud-services-dependencies</artifactId>
            <version>${spring-cloud-services.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-dependencies</artifactId>
            <version>${spring-cloud.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

However, when deployed in PCF it fails to start with an error that seems to come from building the Tracer "Service name must not be null or empty"
Stack Trace (sorry for terrible format)

 Application run failed\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name \'tracingWebClientBeanPostProcessor\' defined in class path resource [io/opentracing/contrib/spring/web/starter/WebClientTracingAutoConfiguration.class]: Unsatisfied dependency expressed through method \'tracingWebClientBeanPostProcessor\' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name \'tracer\' defined in class path resource [io/opentracing/contrib/java/spring/jaeger/starter/JaegerAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.opentracing.Tracer]: Factory method \'tracer\' threw exception; nested exception is java.lang.IllegalArgumentException: Service name must not be null or empty\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:797)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:538)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1176)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:207)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.context.support.PostProcessorRegistrationDelegate.registerBeanPostProcessors(PostProcessorRegistrationDelegate.java:241)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.context.support.AbstractApplicationContext.registerBeanPostProcessors(AbstractApplicationContext.java:723)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:536)\n   2021-01-06T18:51:32.25-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:62)\n   2021-01-06T18:51:32.26-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758)\n   2021-01-06T18:51:32.26-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750)\n   2021-01-06T18:51:32.26-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:405)\n   2021-01-06T18:51:32.26-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.boot.SpringApplication.run(SpringApplication.java:315)\n   2021-01-06T18:51:32.26-0500 [APP/PROC/WEB/0] OUT \tat org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)

What we have tried

• We have been able to run just connected to Redis in PCF while disabling all opentracing.jaeger.enable=false with no issue
• We have been able to run just connected to Jaeger in PCF before bringing in Redis libraries with no issue.
• Added Redis/Jaeger while disabling opentracing.spring.cloud.redis.enabled=false which still throws the same error (This seems odd since bringing in Redis was the suspicion of what would have broken Opentracing)
• Added Redis/Jaeger while systematically disabling other tech one by one until opentracing.spring.cloud.reactor.enabled=false. This seems to eliminate the error and the application is able to start, but we can't figure out why this would have been fine until Redis was added to the mix. Also, we really don't love the idea of disabling this as it leaves a blindspot.

Thought is that the way/order the ReactorTracingAutoConfiguration runs it isn't allowing Spring to have set properties with @value annotation on them. Specifically, the JaegerConfigurationProperties.serviceName field. Also, it seemed odd when looking through many of the TechBlahAutoConfiguration classes in all the starter projects that ReactorTracingAutoConfiguration is missing the @AutoConfigureAfter({TracerAutoConfiguration.class}) annotation that most of the others have. It seems this is only reproducible while deployed in PCF which handles the connection to Redis Tile, but that makes me believe there is a bug in the AutoConfiguration classes here which fire out of order in that scenario.

[dreis2211]

The mentioned auto-configuration registers hooks for some of the reactor parts. Unfortunately, under certain circumstances the hooks there get triggered before the Tracer bean or any of its dependencies like JaegerConfigurationProperties are available. Maybe in your case the Redis autoconfiguration triggers these hooks before a Tracer is available. While the implementation wants to account for that by catching any BeansException, this leaves the JaegerConfigurationProperties unpropagated but already registered in the BeanFactory. Ultimately, this causes errors when JaegerAutoConfiguration tries to initialize the Tracer instance again. This also explains why neither opentracing.jaeger.service.name nor spring.application.name nor any of the other fallbacks work for the service name.

Personally, I just disabled the particular auto-configuration by specifying opentracing.spring.cloud.reactor.enabled=false, but like you I'm not really happy with that.

It seems that Spring-Cloud-Seuth's TraceReactorAutoConfiguration is having another implementation that seems less problematic, but I haven't checked too deep yet if it's having the same problem.