diff --git a/README.md b/README.md index a29c42f..84cac61 100644 --- a/README.md +++ b/README.md @@ -1,269 +1,294 @@ # Maven Index Exporter This Docker image reads a Maven Indexer index and extract information about the indexed documents as a convenient text file. ## Sequence The index files can be dowloaded from any maven repository that uses maven-indexer, like maven central: https://repo1.maven.org/maven2/.index/ Copy all files (i.e. the main index, the updates and properties file) into the volume directory (`$WORKDIR`). It will be mounted as `/work/` in the docker image. The export is then achieved in two steps: * Unpack the Lucene indexes from the Maven Indexer indexes using `maven-indexer-cli`. The command used is: ```` $ java --illegal-access=permit -jar $INDEXER_JAR \ --unpack $FILE_IN \ --destination $WORKDIR/indexes/ \ --type full ```` This generates a set of binary lucene files as shown below: ```` $ ls -lh $WORKDIR/indexes/ total 5,2G -rw-r--r-- 1 root root 500M juil. 7 22:06 _4m.fdt -rw-r--r-- 1 root root 339K juil. 7 22:06 _4m.fdx -rw-r--r-- 1 root root 2,2K juil. 7 22:07 _4m.fnm -rw-r--r-- 1 root root 166M juil. 7 22:07 _4m_Lucene50_0.doc -rw-r--r-- 1 root root 147M juil. 7 22:07 _4m_Lucene50_0.pos [SNIP] -rw-r--r-- 1 root root 363 juil. 7 22:06 _e0.si -rw-r--r-- 1 root root 1,7K juil. 7 22:07 segments_2 -rw-r--r-- 1 root root 8 juil. 7 21:54 timestamp ```` * Export the Lucene documents from the Lucene indexes using `clue`. This generates a set of text files as shown below: ```` $ java --illegal-access=permit -jar $JAR_CLUE $WORKDIR/indexes/ \ export $WORKDIR/export/ text ```` The resulting text export will be available in `$WORKDIR/export/`. ## Output The clue command is documented on [its github page](https://github.com/javasoze/clue). Documents especially are located in the `*.fld` file. A description of the fields used by maven-indexer can be found in the project's API docs: https://maven.apache.org/maven-indexer-archives/maven-indexer-6.0.0/indexer-core/apidocs/org/apache/maven/index/ArtifactInfo.html ## How to build -The build downloads binaries for both tools (maven-indexer-cli and clue). +The build downloads binaries for both tools (maven-indexer-cli and clue), so make sure there is an internet connection. +Go to the `docker/` dorectory and issue the folowing command: ``` $ docker build . -t bbaldassari/maven-index-exporter --no-cache ``` A built and tested image is also available on docker hub at [bbaldassari/maven-index-exporter](https://hub.docker.com/r/bbaldassari/maven-index-exporter). ```` $ docker pull bbaldassari/maven-index-exporter ```` ## How to use The Docker image uses volumes to exchanges files. Prepare a directory with enough space disk (see warning below) and pass it to docker: ``` $ docker run -v /local/work/dir:/work bbaldassari/maven-index-exporter ``` Please note that the local work dir MUST be an absolute path, as docker won't mount relative paths as volumes. For our purpose only the fld file is kept, so if you need other export files you should simply edit the `extract_indexes.sh` script and comment the lines that do the cleaning. ### Size of generated files Beware that maven indexes are compressed and text export can become huge. -When executed on the maven central indexes (5.2 GB), the process generates -49GB of text data on disk: +When executed on the maven central indexes (1.2 GB), the process generates +5.2 GB of intermediate files and 49 GB of final text data on disk: ``` $ du -sh /work/* 49G /work/export 5,2G /work/indexes 1,2G /work/nexus-maven-repository-index.gz ``` -## How to test +## How to test (the quick way) + +There is a bash script called `test_docker_image.sh` in the `resources/` directory, +simply execute it. Tests cover the creation of the docker image, and the results after +execution. + +```` +$ bash test_docker_image.sh +Script started on 20210911_181912. +* Writing log to test_docker_image.log. +* Docker image [maven-index-exporter] doesn't exist. +* Building docker image. +PASS: docker build returned 0. +PASS: Docker image is listed. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has been created. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has 7 docs. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has 26 fields. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has sprova4j-0.1.0-sources.jar. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has sprova4j-0.1.0.pom. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has sprova4j-0.1.1-sources.jar. +PASS: file [/home/boris/Projects/gh_maven-index-exporter/repository_test/export/_1.fld] has sprova4j-0.1.1.pom. +$ +```` + +## How to test (the long road) This repository has a simple, almost-empty maven-indexer index that can be used to test the docker build. To use it, make sure that the directory `repository_test/` is present and run this command: ```` $ docker run -v $(pwd)/repository_test:/work bbaldassari/maven-index-exporter ```` The exported files will be stored in `repository_test/export/`, and output should look like this: ```` $ docker run -v $(pwd)/repository_test:/work bbaldassari/maven-index-exporter Docker Script started on 2021-08-19 09:07:53. # Checks.. * Content of /opt: total 32156 -rw------- 1 root root 18000742 Jan 8 2018 clue-6.2.0-1.0.0.jar -rw-r--r-- 1 root root 2586 Aug 19 08:59 extract_indexes.sh -rw------- 1 root root 14914610 Nov 28 2017 indexer-cli-6.0.0.jar drwxr-xr-x 3 root root 4096 Jun 29 16:23 java * Content of /work: total 36 -rw-r--r-- 1 1000 1000 254 Jul 12 17:43 nexus-maven-repository-index.1.gz -rw-r--r-- 1 1000 1000 32 Jul 12 17:43 nexus-maven-repository-index.1.gz.md5 -rw-r--r-- 1 1000 1000 40 Jul 12 17:43 nexus-maven-repository-index.1.gz.sha1 -rw-r--r-- 1 1000 1000 344 Jul 12 17:43 nexus-maven-repository-index.gz -rw-r--r-- 1 1000 1000 32 Jul 12 17:43 nexus-maven-repository-index.gz.md5 -rw-r--r-- 1 1000 1000 40 Jul 12 17:43 nexus-maven-repository-index.gz.sha1 -rw-r--r-- 1 1000 1000 193 Jul 12 17:43 nexus-maven-repository-index.properties -rw-r--r-- 1 1000 1000 32 Jul 12 17:43 nexus-maven-repository-index.properties.md5 -rw-r--r-- 1 1000 1000 40 Jul 12 17:43 nexus-maven-repository-index.properties.sha1 * Will read files from [/work/nexus-maven-repository-index.gz]. * Found file [/work/nexus-maven-repository-index.gz]. * Found indexer [/opt/indexer-cli-6.0.0.jar]. * Found clue [/opt/clue-6.2.0-1.0.0.jar]. * Java version:. openjdk version "11.0.11" 2021-04-20 OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9) OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode) ############################# Unpacking [/work/nexus-maven-repository-index.gz] to /work/indexes SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Index Folder: /work Output Folder: /work/indexes Total time: 0 sec Final memory: 39M/1004M Unpacking finished on 2021-08-19 09:07:54. ############################# Exporting indexes /work/indexes to /work/export no configuration found, using default configuration Analyzer: class org.apache.lucene.analysis.standard.StandardAnalyzer Query Builder: class com.senseidb.clue.api.DefaultQueryBuilder Directory Builder: class com.senseidb.clue.api.DefaultDirectoryBuilder IndexReader Factory: class com.senseidb.clue.api.DefaultIndexReaderFactory Term Bytesref Display: class com.senseidb.clue.api.StringBytesRefDisplay Payload Bytesref Display: class com.senseidb.clue.api.RawBytesRefDisplay exporting index to text Exporting finished on 2021-08-19 09:07:54. ############################# Cleaning useless files. Size before cleaning: 32.0K /work/export 28.0K /work/indexes 4.0K /work/nexus-maven-repository-index.1.gz 4.0K /work/nexus-maven-repository-index.1.gz.md5 4.0K /work/nexus-maven-repository-index.1.gz.sha1 4.0K /work/nexus-maven-repository-index.gz 4.0K /work/nexus-maven-repository-index.gz.md5 4.0K /work/nexus-maven-repository-index.gz.sha1 4.0K /work/nexus-maven-repository-index.properties 4.0K /work/nexus-maven-repository-index.properties.md5 4.0K /work/nexus-maven-repository-index.properties.sha1 * Removing useless exports. Keeping only fld text extract: total 44K drwxr-xr-x 2 root root 4.0K Aug 19 09:07 export drwxr-xr-x 2 root root 4.0K Aug 19 09:07 indexes -rw-r--r-- 1 1000 1000 254 Jul 12 17:43 nexus-maven-repository-index.1.gz -rw-r--r-- 1 1000 1000 32 Jul 12 17:43 nexus-maven-repository-index.1.gz.md5 -rw-r--r-- 1 1000 1000 40 Jul 12 17:43 nexus-maven-repository-index.1.gz.sha1 -rw-r--r-- 1 1000 1000 344 Jul 12 17:43 nexus-maven-repository-index.gz -rw-r--r-- 1 1000 1000 32 Jul 12 17:43 nexus-maven-repository-index.gz.md5 -rw-r--r-- 1 1000 1000 40 Jul 12 17:43 nexus-maven-repository-index.gz.sha1 -rw-r--r-- 1 1000 1000 193 Jul 12 17:43 nexus-maven-repository-index.properties -rw-r--r-- 1 1000 1000 32 Jul 12 17:43 nexus-maven-repository-index.properties.md5 -rw-r--r-- 1 1000 1000 40 Jul 12 17:43 nexus-maven-repository-index.properties.sha1 Size after cleaning: 8.0K /work/export 28.0K /work/indexes 4.0K /work/nexus-maven-repository-index.1.gz 4.0K /work/nexus-maven-repository-index.1.gz.md5 4.0K /work/nexus-maven-repository-index.1.gz.sha1 4.0K /work/nexus-maven-repository-index.gz 4.0K /work/nexus-maven-repository-index.gz.md5 4.0K /work/nexus-maven-repository-index.gz.sha1 4.0K /work/nexus-maven-repository-index.properties 4.0K /work/nexus-maven-repository-index.properties.md5 4.0K /work/nexus-maven-repository-index.properties.sha1 Make files modifiable by the end-user: Docker Script execution finished on 2021-08-19 09:07:54. ```` The _1.fld file contains the fields for each document: ```` $ head repository_test/export/_1.fld doc 0 field 0 name u type string value al.aldi|sprova4j|0.1.0|sources|jar field 1 name m type string value 1626111735737 field 2 ```` ### Building the test repository The test repository `repository_test` can be rebuilt from the repository_src structure using [indexer-cli](https://search.maven.org/remotecontent?filepath=org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar) with the following commands: ```` $ cd repository_src $ java -jar ~/Downloads/indexer-cli-6.0.0.jar -i index/ -d repository_test/ -r repo1 -s -c SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Repository Folder: /home/boris/Projects/maven-index-exporter/repository_src/repo1 Index Folder: /home/boris/Projects/maven-index-exporter/repository_src/index Output Folder: /home/boris/Projects/maven-index-exporter/repository_src/repository_test Repository name: index Indexers: [min, jarContent] Will create checksum files for all published files (sha1, md5). Will create incremental chunks for changes, along with baseline file. Scanning started Artifacts added: 2 Artifacts deleted: 0 Total time: 1 sec Final memory: 48M/1012M $ java -jar ~/Downloads/indexer-cli-6.0.0.jar -i index/ -d repository_test/ -r repo2 -s -c SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Repository Folder: /home/boris/Projects/maven-index-exporter/repository_src/repo2 Index Folder: /home/boris/Projects/maven-index-exporter/repository_src/index Output Folder: /home/boris/Projects/maven-index-exporter/repository_src/repository_test Repository name: index Indexers: [min, jarContent] Will create checksum files for all published files (sha1, md5). Will create incremental chunks for changes, along with baseline file. Scanning started Artifacts added: 2 Artifacts deleted: 0 Total time: 0 sec Final memory: 7M/1012M $ ```` diff --git a/Dockerfile b/docker/Dockerfile similarity index 100% rename from Dockerfile rename to docker/Dockerfile diff --git a/extract_indexes.sh b/docker/extract_indexes.sh similarity index 100% rename from extract_indexes.sh rename to docker/extract_indexes.sh diff --git a/docker/test_docker_image.log b/docker/test_docker_image.log new file mode 100644 index 0000000..c2232cb --- /dev/null +++ b/docker/test_docker_image.log @@ -0,0 +1,507 @@ +docker run -v ./../repository_test:/work maven-index-exporter +Sending build context to Docker daemon 7.168kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ + + + ---> e67befad95d1 +Step 3/8 : ADD https://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar /opt/ + + + ---> 33c853c55f9c +Step 4/8 : COPY extract_indexes.sh /opt/ + ---> a274735accf4 +Step 5/8 : WORKDIR /work/ + ---> Running in 712968d6bd86 +Removing intermediate container 712968d6bd86 + ---> d6b8480db47c +Step 6/8 : RUN ls /opt/ + ---> Running in c4da658ba366 +clue-6.2.0-1.0.0.jar +extract_indexes.sh +indexer-cli-6.0.0.jar +java +Removing intermediate container c4da658ba366 + ---> cf332d99c1c9 +Step 7/8 : RUN ls -R /work/ + ---> Running in 9a9a3f8c3b88 +/work/: +Removing intermediate container 9a9a3f8c3b88 + ---> 57ac853a063e +Step 8/8 : CMD ["sh", "/opt/extract_indexes.sh", "/work/nexus-maven-repository-index.gz"] + ---> Running in a5a3b17f94fd +Removing intermediate container a5a3b17f94fd + ---> 15f95ea12d1a +Successfully built 15f95ea12d1a +Successfully tagged maven-index-exporter:latest +Sending build context to Docker daemon 8.192kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ +Sending build context to Docker daemon 8.192kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ + + + ---> 9110f82662b7 +Step 3/8 : ADD https://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar /opt/ + + + ---> 5b63d85e1202 +Step 4/8 : COPY extract_indexes.sh /opt/ + ---> d6568520add6 +Step 5/8 : WORKDIR /work/ + ---> Running in 1fc0fa544822 +Removing intermediate container 1fc0fa544822 + ---> ff62bcd081df +Step 6/8 : RUN ls /opt/ + ---> Running in eacc11100b2d +clue-6.2.0-1.0.0.jar +extract_indexes.sh +indexer-cli-6.0.0.jar +java +Removing intermediate container eacc11100b2d + ---> 979ef783a582 +Step 7/8 : RUN ls -R /work/ + ---> Running in 9e5566c9a84a +/work/: +Removing intermediate container 9e5566c9a84a + ---> c7d9f0b30c53 +Step 8/8 : CMD ["sh", "/opt/extract_indexes.sh", "/work/nexus-maven-repository-index.gz"] + ---> Running in 99440f444046 +Removing intermediate container 99440f444046 + ---> 4cf8d38fc66e +Successfully built 4cf8d38fc66e +Successfully tagged maven-index-exporter:latest +Sending build context to Docker daemon 9.728kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ + + + ---> 1a73b1736986 +Step 3/8 : ADD https://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar /opt/ + + + ---> cf76042f7af4 +Step 4/8 : COPY extract_indexes.sh /opt/ + ---> b374045d480d +Step 5/8 : WORKDIR /work/ + ---> Running in d68c71448aa5 +Removing intermediate container d68c71448aa5 + ---> d28dccbe9cd4 +Step 6/8 : RUN ls /opt/ + ---> Running in 310572502db7 +clue-6.2.0-1.0.0.jar +extract_indexes.sh +indexer-cli-6.0.0.jar +java +Removing intermediate container 310572502db7 + ---> 371ddc002d88 +Step 7/8 : RUN ls -R /work/ + ---> Running in 581a932d361e +/work/: +Removing intermediate container 581a932d361e + ---> 07fd66e36f32 +Step 8/8 : CMD ["sh", "/opt/extract_indexes.sh", "/work/nexus-maven-repository-index.gz"] + ---> Running in b97dc3783ba4 +Removing intermediate container b97dc3783ba4 + ---> 475ec388de0f +Successfully built 475ec388de0f +Successfully tagged maven-index-exporter:latest +Docker Script started on 2021-09-11 16:11:38. +# Checks.. +* Content of /opt: +total 32156 +-rw------- 1 root root 18000742 Jan 8 2018 clue-6.2.0-1.0.0.jar +-rw-r--r-- 1 root root 2574 Aug 25 18:28 extract_indexes.sh +-rw------- 1 root root 14914610 Nov 28 2017 indexer-cli-6.0.0.jar +drwxr-xr-x 3 root root 4096 Jun 29 16:23 java +* Content of /work: +total 36 +-rw-r--r-- 1 1000 1000 254 Aug 25 09:24 nexus-maven-repository-index.1.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.1.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.1.gz.sha1 +-rw-r--r-- 1 1000 1000 344 Aug 25 09:24 nexus-maven-repository-index.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.gz.sha1 +-rw-r--r-- 1 1000 1000 193 Aug 25 09:24 nexus-maven-repository-index.properties +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.properties.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.properties.sha1 +* Will read files from [/work/nexus-maven-repository-index.gz]. +* Found file [/work/nexus-maven-repository-index.gz]. +* Found indexer [/opt/indexer-cli-6.0.0.jar]. +* Found clue [/opt/clue-6.2.0-1.0.0.jar]. +* Java version:. +############################# +Unpacking [/work/nexus-maven-repository-index.gz] to /work/indexes +SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". +SLF4J: Defaulting to no-operation (NOP) logger implementation +SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. +Index Folder: /work +Output Folder: /work/indexes +Total time: 0 sec +Final memory: 39M/1004M +Unpacking finished on 2021-09-11 16:11:39. +############################# +Exporting indexes /work/indexes to /work/export +no configuration found, using default configuration +Analyzer: class org.apache.lucene.analysis.standard.StandardAnalyzer +Query Builder: class com.senseidb.clue.api.DefaultQueryBuilder +Directory Builder: class com.senseidb.clue.api.DefaultDirectoryBuilder +IndexReader Factory: class com.senseidb.clue.api.DefaultIndexReaderFactory +Term Bytesref Display: class com.senseidb.clue.api.StringBytesRefDisplay +Payload Bytesref Display: class com.senseidb.clue.api.RawBytesRefDisplay +exporting index to text +Exporting finished on 2021-09-11 16:11:39. +############################# +Cleaning useless files. +Size before cleaning: +32.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Removing useless exports. + Keeping only fld text extract. + Size after cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Make files modifiable by the end-user. +Docker Script execution finished on 2021-09-11 16:11:39. +Sending build context to Docker daemon 14.34kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ +Sending build context to Docker daemon 14.85kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ + + + ---> 908a2a506c00 +Step 3/8 : ADD https://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar /opt/ + + + ---> 40b18ec046eb +Step 4/8 : COPY extract_indexes.sh /opt/ + ---> 85c9dfc61e0f +Step 5/8 : WORKDIR /work/ + ---> Running in 8ab59ca265dd +Removing intermediate container 8ab59ca265dd + ---> f74ef027e618 +Step 6/8 : RUN ls /opt/ + ---> Running in 928e795e7c81 +clue-6.2.0-1.0.0.jar +extract_indexes.sh +indexer-cli-6.0.0.jar +java +Removing intermediate container 928e795e7c81 + ---> 5b78b2951990 +Step 7/8 : RUN ls -R /work/ + ---> Running in d96f9a7dfd47 +/work/: +Removing intermediate container d96f9a7dfd47 + ---> 83a30f3fa55e +Step 8/8 : CMD ["sh", "/opt/extract_indexes.sh", "/work/nexus-maven-repository-index.gz"] + ---> Running in 84ae7dd43899 +Removing intermediate container 84ae7dd43899 + ---> ad0e9f6cec82 +Successfully built ad0e9f6cec82 +Successfully tagged maven-index-exporter:latest +Docker Script started on 2021-09-11 16:18:14. +# Checks.. +* Content of /opt: +total 32156 +-rw------- 1 root root 18000742 Jan 8 2018 clue-6.2.0-1.0.0.jar +-rw-r--r-- 1 root root 2574 Aug 25 18:28 extract_indexes.sh +-rw------- 1 root root 14914610 Nov 28 2017 indexer-cli-6.0.0.jar +drwxr-xr-x 3 root root 4096 Jun 29 16:23 java +* Content of /work: +total 44 +drwxrwxrwx 2 root root 4096 Sep 11 16:11 export +drwxrwxrwx 2 root root 4096 Sep 11 16:11 indexes +-rw-r--r-- 1 1000 1000 254 Aug 25 09:24 nexus-maven-repository-index.1.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.1.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.1.gz.sha1 +-rw-r--r-- 1 1000 1000 344 Aug 25 09:24 nexus-maven-repository-index.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.gz.sha1 +-rw-r--r-- 1 1000 1000 193 Aug 25 09:24 nexus-maven-repository-index.properties +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.properties.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.properties.sha1 +* Will read files from [/work/nexus-maven-repository-index.gz]. +* Found file [/work/nexus-maven-repository-index.gz]. +* Found indexer [/opt/indexer-cli-6.0.0.jar]. +* Found clue [/opt/clue-6.2.0-1.0.0.jar]. +* Java version:. +openjdk version "11.0.11" 2021-04-20 +OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9) +OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode) +############################# +Found /work/indexes, skipping index generation. +28.0K /work/indexes/ +Unpacking finished on 2021-09-11 16:18:14. +############################# +Found /work/export, skipping index export. +total 4K +-rwxrwxrwx 1 root root 1.9K Sep 11 16:11 _1.fld +-rwxrwxrwx 1 root root 0 Sep 11 16:11 write.lock +Exporting finished on 2021-09-11 16:18:14. +############################# +Cleaning useless files. +Size before cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Removing useless exports. + Keeping only fld text extract. + Size after cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Make files modifiable by the end-user. +Docker Script execution finished on 2021-09-11 16:18:14. +Sending build context to Docker daemon 19.46kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ + + + ---> 8ea4a5a0a8c2 +Step 3/8 : ADD https://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar /opt/ + + + ---> 9b4a328cbc1b +Step 4/8 : COPY extract_indexes.sh /opt/ + ---> 2dd17bae7728 +Step 5/8 : WORKDIR /work/ + ---> Running in 595e0fd75c7b +Removing intermediate container 595e0fd75c7b + ---> 305d43bc2d0b +Step 6/8 : RUN ls /opt/ + ---> Running in 9384e58578b9 +clue-6.2.0-1.0.0.jar +extract_indexes.sh +indexer-cli-6.0.0.jar +java +Removing intermediate container 9384e58578b9 + ---> 5f4a4edb19bf +Step 7/8 : RUN ls -R /work/ + ---> Running in ec031f0a2be1 +/work/: +Removing intermediate container ec031f0a2be1 + ---> c5da5e529d72 +Step 8/8 : CMD ["sh", "/opt/extract_indexes.sh", "/work/nexus-maven-repository-index.gz"] + ---> Running in 7f1ad9ee0ff0 +Removing intermediate container 7f1ad9ee0ff0 + ---> 529f1139d2d6 +Successfully built 529f1139d2d6 +Successfully tagged maven-index-exporter:latest +Docker Script started on 2021-09-11 16:19:59. +# Checks.. +* Content of /opt: +total 32156 +-rw------- 1 root root 18000742 Jan 8 2018 clue-6.2.0-1.0.0.jar +-rw-r--r-- 1 root root 2574 Aug 25 18:28 extract_indexes.sh +-rw------- 1 root root 14914610 Nov 28 2017 indexer-cli-6.0.0.jar +drwxr-xr-x 3 root root 4096 Jun 29 16:23 java +* Content of /work: +total 44 +drwxrwxrwx 2 root root 4096 Sep 11 16:11 export +drwxrwxrwx 2 root root 4096 Sep 11 16:11 indexes +-rw-r--r-- 1 1000 1000 254 Aug 25 09:24 nexus-maven-repository-index.1.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.1.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.1.gz.sha1 +-rw-r--r-- 1 1000 1000 344 Aug 25 09:24 nexus-maven-repository-index.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.gz.sha1 +-rw-r--r-- 1 1000 1000 193 Aug 25 09:24 nexus-maven-repository-index.properties +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.properties.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.properties.sha1 +* Will read files from [/work/nexus-maven-repository-index.gz]. +* Found file [/work/nexus-maven-repository-index.gz]. +* Found indexer [/opt/indexer-cli-6.0.0.jar]. +* Found clue [/opt/clue-6.2.0-1.0.0.jar]. +* Java version:. +openjdk version "11.0.11" 2021-04-20 +OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9) +OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode) +############################# +Found /work/indexes, skipping index generation. +28.0K /work/indexes/ +Unpacking finished on 2021-09-11 16:19:59. +############################# +Found /work/export, skipping index export. +total 4K +-rwxrwxrwx 1 root root 1.9K Sep 11 16:11 _1.fld +-rwxrwxrwx 1 root root 0 Sep 11 16:11 write.lock +Exporting finished on 2021-09-11 16:19:59. +############################# +Cleaning useless files. +Size before cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Removing useless exports. + Keeping only fld text extract. + Size after cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Make files modifiable by the end-user. +Docker Script execution finished on 2021-09-11 16:19:59. +Sending build context to Docker daemon 24.06kB +Step 1/8 : FROM adoptopenjdk/openjdk11:alpine-jre + ---> 382f4f02e274 +Step 2/8 : ADD https://github.com/javasoze/clue/releases/download/release-6.2.0-1.0.0/clue-6.2.0-1.0.0.jar /opt/ + + + ---> 58157b61f665 +Step 3/8 : ADD https://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-cli/6.0.0/indexer-cli-6.0.0.jar /opt/ + + + ---> d67d5d21cd81 +Step 4/8 : COPY extract_indexes.sh /opt/ + ---> 19fa37bb0039 +Step 5/8 : WORKDIR /work/ + ---> Running in b2f208c13bd4 +Removing intermediate container b2f208c13bd4 + ---> bd877fbc85bb +Step 6/8 : RUN ls /opt/ + ---> Running in 2fa484fa1543 +clue-6.2.0-1.0.0.jar +extract_indexes.sh +indexer-cli-6.0.0.jar +java +Removing intermediate container 2fa484fa1543 + ---> 6e4e28f585f8 +Step 7/8 : RUN ls -R /work/ + ---> Running in 19f97581c8d4 +/work/: +Removing intermediate container 19f97581c8d4 + ---> 56904223bb09 +Step 8/8 : CMD ["sh", "/opt/extract_indexes.sh", "/work/nexus-maven-repository-index.gz"] + ---> Running in 73b8d2a0143f +Removing intermediate container 73b8d2a0143f + ---> b87c33abb5a9 +Successfully built b87c33abb5a9 +Successfully tagged maven-index-exporter:latest +Docker Script started on 2021-09-11 16:29:15. +# Checks.. +* Content of /opt: +total 32156 +-rw------- 1 root root 18000742 Jan 8 2018 clue-6.2.0-1.0.0.jar +-rw-r--r-- 1 root root 2574 Aug 25 18:28 extract_indexes.sh +-rw------- 1 root root 14914610 Nov 28 2017 indexer-cli-6.0.0.jar +drwxr-xr-x 3 root root 4096 Jun 29 16:23 java +* Content of /work: +total 44 +drwxrwxrwx 2 root root 4096 Sep 11 16:11 export +drwxrwxrwx 2 root root 4096 Sep 11 16:11 indexes +-rw-r--r-- 1 1000 1000 254 Aug 25 09:24 nexus-maven-repository-index.1.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.1.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.1.gz.sha1 +-rw-r--r-- 1 1000 1000 344 Aug 25 09:24 nexus-maven-repository-index.gz +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.gz.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.gz.sha1 +-rw-r--r-- 1 1000 1000 193 Aug 25 09:24 nexus-maven-repository-index.properties +-rw-r--r-- 1 1000 1000 32 Aug 25 09:24 nexus-maven-repository-index.properties.md5 +-rw-r--r-- 1 1000 1000 40 Aug 25 09:24 nexus-maven-repository-index.properties.sha1 +* Will read files from [/work/nexus-maven-repository-index.gz]. +* Found file [/work/nexus-maven-repository-index.gz]. +* Found indexer [/opt/indexer-cli-6.0.0.jar]. +* Found clue [/opt/clue-6.2.0-1.0.0.jar]. +* Java version:. +openjdk version "11.0.11" 2021-04-20 +OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9) +OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode) +############################# +Found /work/indexes, skipping index generation. +28.0K /work/indexes/ +Unpacking finished on 2021-09-11 16:29:15. +############################# +Found /work/export, skipping index export. +total 4K +-rwxrwxrwx 1 root root 1.9K Sep 11 16:11 _1.fld +-rwxrwxrwx 1 root root 0 Sep 11 16:11 write.lock +Exporting finished on 2021-09-11 16:29:15. +############################# +Cleaning useless files. +Size before cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Removing useless exports. + Keeping only fld text extract. + Size after cleaning: +8.0K /work/export +28.0K /work/indexes +4.0K /work/nexus-maven-repository-index.1.gz +4.0K /work/nexus-maven-repository-index.1.gz.md5 +4.0K /work/nexus-maven-repository-index.1.gz.sha1 +4.0K /work/nexus-maven-repository-index.gz +4.0K /work/nexus-maven-repository-index.gz.md5 +4.0K /work/nexus-maven-repository-index.gz.sha1 +4.0K /work/nexus-maven-repository-index.properties +4.0K /work/nexus-maven-repository-index.properties.md5 +4.0K /work/nexus-maven-repository-index.properties.sha1 +* Make files modifiable by the end-user. +Docker Script execution finished on 2021-09-11 16:29:15.