Tomcat Java 10 on Windows CreateJavaVM Failed

I ran into some issues getting the Lucee Tomcat 8.5 service to start on with Java 10.0.2 on a Windows 2016 Server. I was getting errors in the commons-daemon log file like: CreateJavaVM Failed CreateJavaVM Failed The system cannot find the file specified. Failed to start Java, and Failed creating java. My first instinct for this type of error is to make sure that the path to jvm.dll is correct, it was.

Searching for the above errors did not yield any help - many solutions involved copying a msvcr dll files in to the java install, this did not make any difference. They also suggested setting a JAVA_HOME environment variable, this did not help either.

It turns out the problem was that I was it was passing a JVM argument that is not supported on Java 10, in this case it was the java.endorsed.dirs property.

What really led me to the problem was running tomcat from the Command Prompt instead of trying to start it as a service. When I ran: catalina.bat run it outputted the error:

-Djava.endorsed.dirs=C:\lucee\tomcat\endorsed is not supported. Endorsed standards and standalone APIs
in modular form will be supported via the concept of upgradeable modules.

In this case the endorsed folder did not even exist.

Now there was still an issue with Lucee running on Java 10 (or even Java 9) that caused it to throw a java.lang.NoClassDefFoundError: sun/misc/VM exception. The sun.misc.VM class is not found because they moved it to jdk.internal.misc in Java 9+.

The solution for that problem on Lucee is to use Lucee 5.2.9.25+ or 5.3.0.57+. Since 5.2.9 is still in RC at this time there is not an installer, so I had to download the lucee.jar and replace C:\lucee\lib\lucee-5.2.8.50.jar with it.