Skip to content

calling clojure from Java Azure Function ( ExceptionInInitializerError) #55

Closed
Closed
calling clojure from Java Azure Function ( ExceptionInInitializerError)#55
Labels
Bugtriaged
Milestone
Unknown

Description

@kyale-mwendwa
Kyale Mwendwa (kyale-mwendwa)
opened on Feb 20, 2018

I had a previous issue on the same topic described in this thread. I found a solution using maven-assembly-plugin however I encountered the below issue.

Sample code.

1)the following classes

      i) import clojure.java.api.Clojure; ,
     ii) import clojure.lang.IFn;

2)compile your clojure into an uberjar then import it into the java
code.

I have opted for the 2nd option as it's more straight forward.

Here is the clojure code

(ns com.test.app.service
 (:gen-class
	   :name com.test.app.service
	   :main false
	   :methods [^{:static true} [returned [int] int]]))

	(defn returned
	  [number]
	  (* 2 number))

	(defn -returned
	  [number]
	  (returned number))

Here is the Java code.

package com.s.profile;

import java.util.*;
import com.microsoft.azure.serverless.functions.annotation.*;
import com.microsoft.azure.serverless.functions.*;
import com.test.app.service;


/**
 * Azure Functions with HTTP Trigger.
 */
public class Function {
	/**
	 * This function listens at endpoint "/api/hello". Two ways to invoke it using "curl" command in bash:
	 * 1. curl -d "HTTP Body" {your host}/api/hello
	 * 2. curl {your host}/api/hello?name=HTTP%20Query
	 */
	@FunctionName("hello")
	public HttpResponseMessage<String> hello(
			@HttpTrigger(name = "req", methods = {"get", "post"}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
			final ExecutionContext context) {
		context.getLogger().info("Java HTTP trigger processed a request.");

		// Parse query parameter
		String query = request.getQueryParameters().get("name");
		String name = request.getBody().orElse(query);

		if (name == null) {
			return request.createResponse(400, "Please pass a name on the query string or in the request body");
		} else {
			service.returned(4);
			context.getLogger().info("process data" );
			return request.createResponse(200, "Hellos, " + name );
		}
	}
}

And here is the full stacktrace

    info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
		  java.lang.ExceptionInInitializerError
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.Namespace.<init>(Namespace.java:34)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.Namespace.findOrCreate(Namespace.java:176)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.Var.internPrivate(Var.java:151)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.test.app.service.<clinit>(Unknown Source)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.s.profile.Function.hello(Function.java:42)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.lang.reflect.Method.invoke(Method.java:483)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.broker.JavaMethodInvokeInfo.invoke(JavaMethodInvokeInfo.java:22)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.broker.JavaMethodExecutor.execute(JavaMethodExecutor.java:59)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:40)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.handler.MessageHandler.handle(MessageHandler.java:43)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:84)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at com.microsoft.azure.webjobs.script.JavaWorkerClient$StreamingMessagePeer$$Lambda$10/293453555.run(Unknown Source)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:902)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1689)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1644)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
		  Caused by: java.io.FileNotFoundException: Could not locate clojure/core__init.class or clojure/core.clj on classpath:
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.RT.load(RT.java:443)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.RT.load(RT.java:411)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.RT.doInit(RT.java:447)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]
			at clojure.lang.RT.<clinit>(RT.java:329)
	info: Worker.Java.917bfefd-b67a-49b2-b77b-6848c52c3a98[0]

The information I have on that issue is below as per the documentation.

Signals that an unexpected exception has occurred in a static initializer. An ExceptionInInitializerError is thrown to indicate that an exception occurred during evaluation of a static initializer or the initializer for a static variable.

Which static method had a problem the one in my clojure code? Seems unlikely because when I run the code from a more simple "hello world" Java example from the main function it returns fine, which leads me to believe my compiled clojure jar file is not the problem. So where is it stemming from? Are there rules behind how the azure functions work internally that limit what can get called?

Activity

  1. JunyiYi commented on Feb 21, 2018

    @JunyiYi
    J. Yi (JunyiYi)
    on Feb 21, 2018
    Contributor

    The solution should be putting all related libraries into "/lib" folder as specified here. Please notice that this changeset has not been published to Azure yet.

  2. J. Yi (JunyiYi)
    added
    Bug
    triaged
    on Feb 21, 2018

  3. kyale-mwendwa commented on Feb 28, 2018

    @kyale-mwendwa
    Kyale Mwendwa (kyale-mwendwa)
    on Feb 28, 2018
    Author

    This doesn't seem to work either.

    Below is my pom.xml

    `


    4.0.0

    <groupId>profile001</groupId>
<artifactId>profile001</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>Azure Java Functions</name>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <functionAppName>-20180130112655795</functionAppName>
    <functionAppRegion>westus</functionAppRegion>
</properties>

<dependencies>
    <dependency>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>azure-functions-java-core</artifactId>
        <version>[1.0.0-beta-1,1.0.0)</version>
    </dependency>

    <!-- Test -->
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.12</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.mockito</groupId>
        <artifactId>mockito-core</artifactId>
        <version>2.4.0</version>
        <scope>test</scope>
    </dependency>
	
    <dependency>
        <groupId>com.test.app.service</groupId>
        <artifactId>com.test.app.service</artifactId>
        <version>1.0.1</version>
    </dependency>
</dependencies>

<build>
    <pluginManagement>
        <plugins>
            <plugin>
                <artifactId>maven-resources-plugin</artifactId>
                <version>3.0.2</version>
            </plugin>
            <plugin>
                <groupId>com.microsoft.azure</groupId>
                <artifactId>azure-functions-maven-plugin</artifactId>
                <version>0.1.10</version>
            </plugin>
        </plugins>
    </pluginManagement>

    <plugins>
        <plugin>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-functions-maven-plugin</artifactId>
            <configuration>
                <resourceGroup>java-functions-group</resourceGroup>
                <appName>${functionAppName}</appName>
                <region>${functionAppRegion}</region>
                <appSettings>
                    <property>
                        <name>FUNCTIONS_EXTENSION_VERSION</name>
                        <value>beta</value>
                    </property>
                </appSettings>
            </configuration>
            <executions>
                <execution>
                    <id>package-functions</id>
                    <goals>
                        <goal>package</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <artifactId>maven-resources-plugin</artifactId>
            <executions>
                <execution>
                    <id>copy-resources</id>
                    <phase>package</phase>
                    <goals>
                        <goal>copy-resources</goal>
                    </goals>
                    <configuration>
                        <overwrite>true</overwrite>
                        <outputDirectory>${project.build.directory}/azure-functions/${functionAppName}
                        </outputDirectory>
                        <resources>
                            <resource>
                                <directory>${project.basedir}</directory>
                                <includes>
                                    <include>host.json</include>
                                    <include>local.settings.json</include>
                                </includes>
                            </resource>
                        </resources>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        
        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <configuration>
               <outputDirectory>${project.build.directory}/azure-functions/${functionAppName}</outputDirectory>
               <appendAssemblyId>false</appendAssemblyId>
               <descriptorRefs>
                  <descriptorRef>jar-with-dependencies</descriptorRef>
               </descriptorRefs>
               <archive />
            </configuration>
            <executions>
               <execution>
                  <id>make-assembly</id>
                  <phase>package</phase>
                  <goals>
                     <goal>assembly</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
         
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>3.0.2</version>
            <executions>
              <execution>
                <id>copy-dependencies</id>
                <phase>prepare-package</phase>
                <goals>
                  <goal>copy-dependencies</goal>
                </goals>
                <configuration>
                  <outputDirectory>${project.build.directory}/azure-functions/${functionAppName}/lib</outputDirectory>
                  <overWriteReleases>false</overWriteReleases>
                  <overWriteSnapshots>false</overWriteSnapshots>
                  <overWriteIfNewer>true</overWriteIfNewer>
                </configuration>
              </execution>
            </executions>
        </plugin>
        
    </plugins>

</build>
    `

    If i try remove "maven-assembly-plugin" then it reverts to the error "java.lang.NoClassDefFoundError". The strange part is if i add the following line of code

    String t = String.valueOf(mpesa.returned(2));
    context.getLogger().info("process data"+ t);

    And run the command

    mvn clean package

    I see the following

    T E S T S

    Running com.saida.profile.FunctionTest
    Feb 28, 2018 11:01:02 AM com.saida.profile.Function hello
    INFO: Java HTTP trigger processed a request.
    Feb 28, 2018 11:01:03 AM com.saida.profile.Function hello
    INFO: before process data
    Feb 28, 2018 11:01:08 AM com.saida.profile.Function hello
    INFO: process data4
    plus: 4
    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.674 sec

    Results :

    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

    Which means that it is able to see the jar file at this point.

    So what is going wrong when running the service?

  4. esuomi commented on Mar 5, 2018

    @esuomi
    Esko Suomi (esuomi)
    on Mar 5, 2018

    I spent some time today digging into this as I also need to run Clojure on Azure Functions and it seems the issue is caused by clojure.lang.RT not respecting the loading rules of java.net.URLClassLoader. This is documented rather poorly in some ancient tomes such as here: https://dev.clojure.org/jira/browse/CLJ-260

    As can be seen from this project's code, it is currently hardcoded to use URLClassLoader through its com.microsoft.azure.webjobs.script.reflect.DefaultClassLoaderProvider class which is directly instantiated in com.microsoft.azure.webjobs.script.JavaWorkerClient#26 and finally used in com.microsoft.azure.webjobs.script.broker.JavaMethodExecutor.java#L54-L57

    Apparently due to this mismatch Clojure natively has clojure.lang.DynamicClassLoader which I guess is what should be used instead of just plain URLClassLoader. This guess is based on this StackOverflow answer.

    Going on with the assumption that this is caused by wrong kind of ClassLoader it also explains why resetting the ClassLoader (as seen here) by putting Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader()); into code doesn't work either. And in pure-Clojure environment this wouldn't work anyways as there would then need to be a Java class just for resetting classloader which isn't as straightforward as it should...

    So, assuming this is a ClassLoader issue we'd need to be able to override the ClassLoader used by Azure Functions so that the JavaWorkerClient would use user provided ClassLoaderProvider instead of the hardcoded one using some reasonable mechanism. Then we'd only need to write ClojureAwareClassLoaderProvider and initialise clojure.lang.DynamicClassLoader from there and that's it.

    Maybe.

  5. JunyiYi commented on Apr 4, 2018

    @JunyiYi
    J. Yi (JunyiYi)
    on Apr 4, 2018
    Contributor

    Esko Suomi (@esuomi) , there is a chicken-and-egg problem of customized class loader: you don't have any chance to specify how to load your class before your class is loaded.

  6. esuomi commented on Apr 5, 2018

    @esuomi
    Esko Suomi (esuomi)
    on Apr 5, 2018

    J. Yi (@JunyiYi) Well yes, which is why one generally should avoid customizing it and let the platform provide eg. context or system classloader and trust they do the right thing... :) ClassLoaders (and to extent bytecode generation) have always been a mess, especially after the advent of JVM as platform for alternative languages.

  7. krukow commented on Apr 19, 2018

    @krukow
    Karl Krukow (krukow)
    on Apr 19, 2018

    J. Yi (@JunyiYi) or Esko Suomi (@esuomi) - is there any progress on this or any news about prioritizing a solution?

  8. esuomi commented on Apr 20, 2018

    @esuomi
    Esko Suomi (esuomi)
    on Apr 20, 2018

    Karl Krukow (@krukow) I'd be interested as well - I'm a user, not MS employee :) If this was my project, I'd remove the whole classloader thing and instead replace it with JVM startup script which uses Java classpath wildcard feature to pick the dependencies from specific subdir which would circumvent the entire problem but this is just me - there probably is some other reason why this isn't done in that way but instead the classloader based solution has been chosen, for example maybe the ultimate goal is to use shared JVMs for multiple functions independent from each other. Of course being a relatively old Java user I remember the days of OSGi and other "ClassWorld" systems and I'm not excited about that at all, but now I'm just rambling... :)

  9. krukow commented on Apr 20, 2018

    @krukow
    Karl Krukow (krukow)
    on Apr 20, 2018

    Do we know what AWS is doing?

  10. esuomi commented on Apr 21, 2018

    @esuomi
    Esko Suomi (esuomi)
    on Apr 21, 2018

    I'd imagine they're doing pretty much what I said as that is the sensible thing to do but that's just an educated guess. I haven't really used AWS Lambda in such extent that I could claim to know for sure.

  11. JunyiYi commented on May 17, 2018

    @JunyiYi
    J. Yi (JunyiYi)
    on May 17, 2018
    Contributor

    Esko Suomi (@esuomi) Karl Krukow (@krukow) , we do not use JVM arguments because the worker doesn't know anything about the functions (not even the jar path) when it is started, after sometime the server will tell the worker to load some specific jar file and run the functions in there.

    Java worker will not create any new processes due to the suggestion from the host side. We can create new threads, but we are not supposed to create processes. So the host can easily terminate and restart everything by just killing the worker process.

  12. Brett Samblanet (brettsam)
    added this to the unknown milestone on May 30, 2018

  13. pragnagopa commented on Aug 5, 2019

    @pragnagopa
    Pragna Gopa (pragnagopa)
    on Aug 5, 2019
    Member

    Kyale Mwendwa (@kyale-mwendwa) - we have made several changes / improvements on we load dependencies for java functions. Can you please let us know if this still reproes?

  14. shreyas-gopalakrishna commented on Aug 10, 2022

    @shreyas-gopalakrishna
    Shreyas Gopalakrishna (shreyas-gopalakrishna)
    on Aug 10, 2022
    Member

    Closing this issue due to inactivity. Java worker has been improved with several changes. Please reopen if this issue persists and we can help you 😄

  15. Shreyas Gopalakrishna (shreyas-gopalakrishna)
    closed this as completedon Aug 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bugtriaged

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions