Compiling Kotlin Code in Tests

Imagine we have writte a Kotlin code generator—e.g. with the excellent kotlin poet. Now we want to write a test that checks that the generated code actually compiles. To compile Kotlin code, we need the Kotlin compiler dependency in our project. It contains the K2JVMCompiler^†throughout this post, we’ll target the JVM class which allows us to compile our generated code.

testCompile("org.jetbrains.kotlin:kotlin-compiler:$kotlinVersion")

<dependency>
	<groupId>org.jetbrains.kotlin</groupId>
	<artifactId>kotlin-compiler</artifactId>
	<version>${kotlin-version}</version>
</dependency>

TL;DR

If you want to skip the explanations, you can jump directly to the source code:

• KotlinTestCompiler.kt
• KotlinTestCompilerLoggingMessageCollector.kt

Setting the Input

We configure K2JVMCompiler quite similar to how we call the compiler from the command line. This means we provide our source files as free arguments—arguments without a flag. Usually, we’ll just provide the source root into which we generated or source files:

val compilerArgs = K2JVMCompilerArguments().apply {
	freeArgs += sourceDirectory.toString()
}

Setting up the Classpath

Usually, our code will depend on other types, which have to be on the classpath when compiling. The users of our generator will provide the types through their dependency management system of choice, like Gradle or Maven. We, on the other hand, will likely already have all necessary dependencies on the classpath which is used to execute the tests. So using the runtime classpath is an intuitive solution here.

Additionally, we have to tell the compiler that it should not try to gather the Kotlin stdlib and reflect modules from our Kotlin home. First, we do not know where the Kotlin home is on any given machine that runs our tests. Second, we already have these dependencies on the classpath anyway, so there is no use in getting them from the Kotlin home.

val compilerArgs = K2JVMCompilerArguments().apply {
	freeArgs += sourceDirectory.toString()
	classpath = System.getProperty("java.class.path")
	noStdlib = true
	noReflect = true
}

What if the code has a dependency that is not on the test classpath yet? Figuring out where the relevant JAR might be located on any given machine is practically impossible. In such cases, I would simply put the missing dependency on test runtime classpath, using Gradle’s testRuntime or Maven’s <scope>test</scope>.

Handling Errors

What happens on compilation errors or warnings? For tests, we want several things to happen:

• The test should fail on errors—sometimes even on warnings.
• Errors and warnings should be logged in a way that they’re easy to debug. Ideally, the messages contain excerpts of the source code.
• All found errors and warnings should be logged, to not have to re-run tests just to find the next errors.

We can achieve this behaviour by implementing our own MessageCollector. I have written an implementation that has all of the aforementioned properties: KotlinTestCompilerLoggingMessageCollector. It produces output like this one:

LOGGING: Using Kotlin home directory <no_path>

LOGGING: Configuring the compilation environment

ERROR: Unresolved reference: x
@ /tmp/test4540766204487691529/Test.kt (3:8):
> class Test {
>     fun testMethod(): String {
>         this.x = 8
> -------------^
>         return 5
>     }

ERROR: The integer literal does not conform to the expected type String
@ /tmp/test4540766204487691529/Test.kt (4:10):
>     fun testMethod(): String {
>         this.x = 8
>         return 5
> ---------------^
>     }
> }

Putting it all together

After all these considerations, we can write our helper method that checks whether generated Kotlin code compiles:

package io.metamodel.generator

/* imports */

object KotlinTestCompiler {

	/**
	 * Uses the classpath of the running JVM to compile all files in the
	 * provided [directory]. Fails if the compiler outputs any message with a
	 * severity as bad as or worse than the [failOn] severity.
	 */
	fun assertKotlinCompiles(
		directory: Path,
 		failOn: CompilerMessageSeverity = CompilerMessageSeverity.ERROR
	) {
		val compilerArgs = K2JVMCompilerArguments().apply {
			freeArgs += directory.toString()
			classpath = System.getProperty("java.class.path")
			noStdlib = true
			noReflect = true
		}
		val errorCollector = KotlinTestCompilerLoggingMessageCollector(failOn)
		K2JVMCompiler().exec(errorCollector, Services.EMPTY, compilerArgs)
		if (errorCollector.hasErrors()) {
			Assertions.fail<Void>("Compilation failed: ${errorCollector.errorMessage} (see output above)")
		}
	}
}