Automating Docker deployments

https://ideas.into.software/engineering/2019/04/28/osgi-r7-docker-automation.html

Having our code modularized, select modules already communicating asynchronously via a messaging solution of our choice, transitioning the Monolith application to work in a cluster is simply a matter of deciding which of those modules should comprise separate containers.

With automation tools used in this complete application https://github.com/ideas-into-software/automated-linguistic-analysis available for cloning and deploying yourself, some of which I will describe in more detail in this article, much of the mundane, repetitive tasks can be configured once and executed in a matter of seconds during Maven's install and deploy phases.

1. Let's say we decide that our automated transcription https://github.com/ideas-into-software/automated-linguistic-analysis/tree/master/service-transcription-impl and linguistic analysis https://github.com/ideas-into-software/automated-linguistic-analysis/tree/master/service-linguistics-impl modules, executing potentially long running operations, should be extracted into separate containers so one or more instances of each can be deployed in a cluster. Using OSGi enRoute Maven 'application' archetype[1] we create a separate application for each, i.e.:

 mvn archetype:generate -DarchetypeGroupId=org.osgi.enroute.archetype \
    -DarchetypeArtifactId=application \
    -DgroupId=software.into.ala \
    -DartifactId=k8-transcription-app \
    -Dversion=0.0.1-SNAPSHOT \
    -Dpackage=software.into.ala \
    -Dimpl-artifactId=service-transcription-impl \
    -Dimpl-groupId=software.into.ala \
    -Dimpl-version=0.0.1-SNAPSHOT \
    -Dapp-target-java-version=8 

and

 mvn archetype:generate -DarchetypeGroupId=org.osgi.enroute.archetype \
   -DarchetypeArtifactId=application \
   -DgroupId=software.into.ala \
   -DartifactId=k8-linguistics-app \
   -Dversion=0.0.1-SNAPSHOT \
   -Dpackage=software.into.ala \
   -Dimpl-artifactId=service-linguistics-impl \
   -Dimpl-groupId=software.into.ala \
   -Dimpl-version=0.0.1-SNAPSHOT \
   -Dapp-target-java-version=8

2. This gives us two new modules–k8-transcription-app https://github.com/ideas-into-software/automated-linguistic-analysis/tree/master/k8-transcription-app and k8-linguistics-app https://github.com/ideas-into-software/automated-linguistic-analysis/tree/master/k8-linguistics-app–each comprised of run descriptors and OSGi R7 Configurator configuration.json files. We then adjust POMs and run descriptor files in each so that dependencies required can be resolved properly; we also address the quirks of JPA, Hibernate and Camel "Using Camel and RabbitMQ in an OSGi R7 application, including custom message types" via additional instructions in run descriptor files.

3. Obviously, since our Monolith application consists of more than just the transcription and linguistics modules, we pull the user facing functionality into a separate application called k8-web-app https://github.com/ideas-into-software/automated-linguistic-analysis/tree/master/k8-web-app. We use the same Maven 'application' archetype and apply the same steps as just described in step no. 2.

4. Since we'll be running these in a cluster, we'll need an image for each, and we prefer to use Docker for handling this. Because it would be a terrible waste of time to switch between Maven and e.g. shell script or be manually executing commands each time we want to run the latest versions of each of these apps in a cluster, we'd like to sync with our Maven build both the building of Docker images and pushing those to registry, with no additional commands to be entered or switching to other tools. For this we use the k8-maven-plugin[2], which can handle all this and more–the “more” I will describe in the next article, when discussing automation of cluster deployment.

5. I like tools which allow to do easy things quickly and then perhaps also have options for more advanced stuff. I evaluated several Maven plugins for both Docker and Kubernetes automation and the k8-maven-plugin is the only plugin which currently fits these criteria. Unfortunately, it was missing some of the other stuff I needed to achieve a fully automated solution, therefore I forked and added these, and this is the version I'm using[3]; until this new version is released, you can clone and install it into your local repository:

mvn install:install-file -Dfile=target/k8-maven-plugin-1.0.5.jar -DgroupId=com.github.deanjameseverett -DartifactId=k8-maven-plugin -Dversion=1.0.5 -Dpackaging=jar

6. We now include the plugin in our Maven build. We start with the main project POM https://github.com/ideas-into-software/automated-linguistic-analysis/blob/master/pom.xml by extracting into variables some of the properties we'll refer to later, e.g. the Docker registry being used–for local development, we can use local Docker registry[4]–i.e.:

<properties> 
  (…) 
  <k8-maven.docker-registry>192.168.0.53:5000</k8-maven.docker-registry>
  (…)
</properties>

7. Since we'd like to trigger building of Docker images explicitly, and not have this happen every time we build our application, we create a new Maven profile, and there we configure the plugin, i.e.

   <profile>
     <id>k8</id>
     <build>
       <pluginManagement>
         <plugins>
           (…)
           <plugin>
             <groupId>com.github.deanjameseverett</groupId>
             <artifactId>k8-maven-plugin</artifactId>
             <version>${k8-maven.version}</version>
             <executions>
               <execution>
                 <id>container</id>
                 <phase>install</phase>
                 <goals>
                   <goal>deleteImage</goal>
                   <goal>buildImage</goal>
                 </goals>
                 <configuration>
                   <dockerRegistry>${k8-maven.docker-registry}</dockerRegistry>
                   <imageName>ala-${project.artifactId}</imageName>
                 </configuration>                   
               </execution>
             (…) 
             </executions>
           </plugin>
         </plugins>
       </pluginManagement>
     </build>
   </profile>

8. For each application extracted in steps 1-3, we now add Docker files. The convention to be used for this plugin is to have a Docker file, as well as any other files to be included in the Docker image–aside from the application itself–in the src/main/docker directory https://github.com/ideas-into-software/automated-linguistic-analysis/tree/master/k8-web-app/src/main/docker. Since we can now use Maven variables inside Docker files, instead of hard-coding any of the info, we can simply duplicate same Docker file definition three times, i.e.:

FROM jeanblanchard/java:8
COPY @project.artifactId@.jar /app/ala-@project.artifactId@.jar
COPY logback.xml /app
WORKDIR /app
ENTRYPOINT ["java","-jar","/app/ala-@project.artifactId@.jar"]

9. We can now build and push to registry Docker images containing latest versions of our applications with one simple command:

mvn -P k8 install

---

[1]: “OSGi enRoute Archetypes“ https://enroute.osgi.org/about/112-enRoute-Archetypes.html

[2]: “Kubernetes Maven Plugin“ https://github.com/deanjameseverett/k8-maven-plugin

[3]: “Kubernetes Maven Plugin (fork)“ https://github.com/ideas-into-software/k8-maven-plugin

[4]: “Deploy a registry server“ https://docs.docker.com/registry/deploying/