TL;DR

I try to remote debug Apache Drill in Embedded Mode with IntelliJ on Windows10.

Prerequirements

Apache Drill installation
Apache Drill source code(same version)
IDEA IntelliJ

Set DRILL_SHELL_JAVA_OPTS

Using sqlline.bat for starting drill as embedded mode. We can set debug options to DRILL_SHELL_JAVA_OPTS.

1set DRILL_SHELL_JAVA_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000

Timeout Configuration

To avoid timeout for debugging, set drill.exec.rpc.user.timeout to 0 in drill-override.conf.

drill-override.conf:

1drill.exec: {
2  cluster-id: "drillbits1",
3  zk.connect: "localhost:2181",
4  rpc: {
5    user: {
6      timeout: 0
7    }
8  }
9}

Set breakpoints

In IntelliJ, set breakpoints that you would like to check in source code. If you don’t have such as points, I recommend to set it in Foreman constructor.

Create debuggin configuration in IntelliJ

Select menu Run -> Edit Configuration.
Click + on Remote to create new debug configuration for drill. Then change configuration name and dubugging port.

Start drill embedded mode

In command prompt, run sqlline.bat -u "jdbc:drill:zk=local" to start sqlline.

1D:\development\drill\apache-drill-1.10.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
2DRILL_ARGS - " -u jdbc:drill:zk=local"
3HADOOP_HOME not detected...
4HBASE_HOME not detected...
5Calculating Drill classpath...
6Listening for transport dt_socket at address: 8000

Start to debug on IntelliJ

Select menu Run -> Debug 'drill' to start debug drill. Below messages are shown in IntelliJ console.

1Connected to the target VM, address: 'localhost:8000', transport: 'socket'

Then sqlline starts to wait for SQL input.

1apache drill 1.10.0
2"json ain't no thang"
30: jdbc:drill:zk=local>

Conclusion

Getting ready for debugging drill. Enjoy it!

C:\development\java\jdk1.8.0_181\bin\java -Didea.launcher.port=7532 “-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 15.0.4\bin” -Dfile.encoding=UTF-8 -classpath “C:\development\java\jdk1.8.0_181\jre\lib\charsets.jar;C:\development\java\jdk1.8.0_181\jre\lib\deploy.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\access-bridge-64.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\cldrdata.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\dnsns.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\jaccess.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\jfxrt.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\localedata.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\nashorn.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\sunec.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\sunjce_provider.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\sunmscapi.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\sunpkcs11.jar;C:\development\java\jdk1.8.0_181\jre\lib\ext\zipfs.jar;C:\development\java\jdk1.8.0_181\jre\lib\javaws.jar;C:\development\java\jdk1.8.0_181\jre\lib\jce.jar;C:\development\java\jdk1.8.0_181\jre\lib\jfr.jar;C:\development\java\jdk1.8.0_181\jre\lib\jfxswt.jar;C:\development\java\jdk1.8.0_181\jre\lib\jsse.jar;C:\development\java\jdk1.8.0_181\jre\lib\management-agent.jar;C:\development\java\jdk1.8.0_181\jre\lib\plugin.jar;C:\development\java\jdk1.8.0_181\jre\lib\resources.jar;C:\development\java\jdk1.8.0_181\jre\lib\rt.jar;D:\development\repository\git\parquet-mr\parquet-tools\target\classes;D:\development\repository\git\parquet-mr\parquet-format-structures\target\classes;D:\development\eclipse\mars.m2\repository\org\slf4j\slf4j-api\1.7.22\slf4j-api-1.7.22.jar;D:\development\eclipse\mars.m2\repository\org\apache\thrift\libthrift\0.9.3\libthrift-0.9.3.jar;D:\development\eclipse\mars.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar;D:\development\repository\git\parquet-mr\parquet-hadoop\target\classes;D:\development\repository\git\parquet-mr\parquet-column\target\classes;D:\development\repository\git\parquet-mr\parquet-common\target\classes;D:\development\eclipse\mars.m2\repository\org\apache\yetus\audience-annotations\0.7.0\audience-annotations-0.7.0.jar;D:\development\repository\git\parquet-mr\parquet-encoding\target\classes;D:\development\eclipse\mars.m2\repository\it\unimi\dsi\fastutil\7.0.13\fastutil-7.0.13.jar;D:\development\repository\git\parquet-mr\parquet-jackson\target\classes;D:\development\eclipse\mars.m2\repository\org\codehaus\jackson\jackson-mapper-asl\1.9.13\jackson-mapper-asl-1.9.13.jar;D:\development\eclipse\mars.m2\repository\org\codehaus\jackson\jackson-core-asl\1.9.13\jackson-core-asl-1.9.13.jar;D:\development\eclipse\mars.m2\repository\org\xerial\snappy\snappy-java\1.1.2.6\snappy-java-1.1.2.6.jar;D:\development\eclipse\mars.m2\repository\commons-pool\commons-pool\1.6\commons-pool-1.6.jar;D:\development\eclipse\mars.m2\repository\org\mortbay\jetty\jetty-util\6.1.26\jetty-util-6.1.26.jar;D:\development\eclipse\mars.m2\repository\xmlenc\xmlenc\0.52\xmlenc-0.52.jar;D:\development\eclipse\mars.m2\repository\org\apache\htrace\htrace-core\3.1.0-incubating\htrace-core-3.1.0-incubating.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-yarn-api\2.7.3\hadoop-yarn-api-2.7.3.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-annotations\2.7.3\hadoop-annotations-2.7.3.jar;D:\development\eclipse\mars.m2\repository\commons-cli\commons-cli\1.3.1\commons-cli-1.3.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-common\3.1.1\hadoop-common-3.1.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\commons\commons-math3\3.1.1\commons-math3-3.1.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\httpcomponents\httpclient\4.5.2\httpclient-4.5.2.jar;D:\development\eclipse\mars.m2\repository\commons-codec\commons-codec\1.11\commons-codec-1.11.jar;D:\development\eclipse\mars.m2\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;D:\development\eclipse\mars.m2\repository\commons-net\commons-net\3.6\commons-net-3.6.jar;D:\development\eclipse\mars.m2\repository\commons-collections\commons-collections\3.2.2\commons-collections-3.2.2.jar;D:\development\eclipse\mars.m2\repository\javax\servlet\javax.servlet-api\3.1.0\javax.servlet-api-3.1.0.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-server\9.3.19.v20170502\jetty-server-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-http\9.3.19.v20170502\jetty-http-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-io\9.3.19.v20170502\jetty-io-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-util\9.3.19.v20170502\jetty-util-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-servlet\9.3.19.v20170502\jetty-servlet-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-security\9.3.19.v20170502\jetty-security-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-webapp\9.3.19.v20170502\jetty-webapp-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\org\eclipse\jetty\jetty-xml\9.3.19.v20170502\jetty-xml-9.3.19.v20170502.jar;D:\development\eclipse\mars.m2\repository\javax\servlet\jsp\jsp-api\2.1\jsp-api-2.1.jar;D:\development\eclipse\mars.m2\repository\com\sun\jersey\jersey-core\1.19\jersey-core-1.19.jar;D:\development\eclipse\mars.m2\repository\javax\ws\rs\jsr311-api\1.1.1\jsr311-api-1.1.1.jar;D:\development\eclipse\mars.m2\repository\com\sun\jersey\jersey-servlet\1.19\jersey-servlet-1.19.jar;D:\development\eclipse\mars.m2\repository\com\sun\jersey\jersey-json\1.19\jersey-json-1.19.jar;D:\development\eclipse\mars.m2\repository\org\codehaus\jettison\jettison\1.1\jettison-1.1.jar;D:\development\eclipse\mars.m2\repository\com\sun\xml\bind\jaxb-impl\2.2.3-1\jaxb-impl-2.2.3-1.jar;D:\development\eclipse\mars.m2\repository\org\codehaus\jackson\jackson-jaxrs\1.9.2\jackson-jaxrs-1.9.2.jar;D:\development\eclipse\mars.m2\repository\org\codehaus\jackson\jackson-xc\1.9.2\jackson-xc-1.9.2.jar;D:\development\eclipse\mars.m2\repository\com\sun\jersey\jersey-server\1.19\jersey-server-1.19.jar;D:\development\eclipse\mars.m2\repository\commons-logging\commons-logging\1.1.3\commons-logging-1.1.3.jar;D:\development\eclipse\mars.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar;D:\development\eclipse\mars.m2\repository\commons-lang\commons-lang\2.6\commons-lang-2.6.jar;D:\development\eclipse\mars.m2\repository\commons-beanutils\commons-beanutils\1.9.3\commons-beanutils-1.9.3.jar;D:\development\eclipse\mars.m2\repository\org\apache\commons\commons-configuration2\2.1.1\commons-configuration2-2.1.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\commons\commons-lang3\3.4\commons-lang3-3.4.jar;D:\development\eclipse\mars.m2\repository\org\apache\avro\avro\1.7.7\avro-1.7.7.jar;D:\development\eclipse\mars.m2\repository\com\thoughtworks\paranamer\paranamer\2.3\paranamer-2.3.jar;D:\development\eclipse\mars.m2\repository\com\google\re2j\re2j\1.1\re2j-1.1.jar;D:\development\eclipse\mars.m2\repository\com\google\protobuf\protobuf-java\2.5.0\protobuf-java-2.5.0.jar;D:\development\eclipse\mars.m2\repository\com\google\code\gson\gson\2.2.4\gson-2.2.4.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-auth\3.1.1\hadoop-auth-3.1.1.jar;D:\development\eclipse\mars.m2\repository\com\nimbusds\nimbus-jose-jwt\4.41.1\nimbus-jose-jwt-4.41.1.jar;D:\development\eclipse\mars.m2\repository\com\github\stephenc\jcip\jcip-annotations\1.0-1\jcip-annotations-1.0-1.jar;D:\development\eclipse\mars.m2\repository\net\minidev\json-smart\2.3\json-smart-2.3.jar;D:\development\eclipse\mars.m2\repository\net\minidev\accessors-smart\1.2\accessors-smart-1.2.jar;D:\development\eclipse\mars.m2\repository\org\ow2\asm\asm\5.0.4\asm-5.0.4.jar;D:\development\eclipse\mars.m2\repository\org\apache\curator\curator-framework\2.12.0\curator-framework-2.12.0.jar;D:\development\eclipse\mars.m2\repository\com\jcraft\jsch\0.1.54\jsch-0.1.54.jar;D:\development\eclipse\mars.m2\repository\org\apache\curator\curator-client\2.12.0\curator-client-2.12.0.jar;D:\development\eclipse\mars.m2\repository\org\apache\curator\curator-recipes\2.12.0\curator-recipes-2.12.0.jar;D:\development\eclipse\mars.m2\repository\com\google\code\findbugs\jsr305\3.0.0\jsr305-3.0.0.jar;D:\development\eclipse\mars.m2\repository\org\apache\htrace\htrace-core4\4.1.0-incubating\htrace-core4-4.1.0-incubating.jar;D:\development\eclipse\mars.m2\repository\org\apache\zookeeper\zookeeper\3.4.9\zookeeper-3.4.9.jar;D:\development\eclipse\mars.m2\repository\jline\jline\0.9.94\jline-0.9.94.jar;D:\development\eclipse\mars.m2\repository\org\apache\commons\commons-compress\1.4.1\commons-compress-1.4.1.jar;D:\development\eclipse\mars.m2\repository\org\tukaani\xz\1.0\xz-1.0.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-simplekdc\1.0.1\kerb-simplekdc-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-client\1.0.1\kerb-client-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerby-config\1.0.1\kerby-config-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-core\1.0.1\kerb-core-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerby-pkix\1.0.1\kerby-pkix-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerby-asn1\1.0.1\kerby-asn1-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerby-util\1.0.1\kerby-util-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-common\1.0.1\kerb-common-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-crypto\1.0.1\kerb-crypto-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-util\1.0.1\kerb-util-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\token-provider\1.0.1\token-provider-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-admin\1.0.1\kerb-admin-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-server\1.0.1\kerb-server-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerb-identity\1.0.1\kerb-identity-1.0.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\kerby\kerby-xdr\1.0.1\kerby-xdr-1.0.1.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.7.8\jackson-databind-2.7.8.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.7.0\jackson-annotations-2.7.0.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\jackson\core\jackson-core\2.7.8\jackson-core-2.7.8.jar;D:\development\eclipse\mars.m2\repository\org\codehaus\woodstox\stax2-api\3.1.4\stax2-api-3.1.4.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\woodstox\woodstox-core\5.0.3\woodstox-core-5.0.3.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-core\3.1.1\hadoop-mapreduce-client-core-3.1.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-yarn-client\3.1.1\hadoop-yarn-client-3.1.1.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-yarn-common\3.1.1\hadoop-yarn-common-3.1.1.jar;D:\development\eclipse\mars.m2\repository\javax\xml\bind\jaxb-api\2.2.11\jaxb-api-2.2.11.jar;D:\development\eclipse\mars.m2\repository\com\sun\jersey\jersey-client\1.19\jersey-client-1.19.jar;D:\development\eclipse\mars.m2\repository\com\google\inject\guice\4.0\guice-4.0.jar;D:\development\eclipse\mars.m2\repository\javax\inject\javax.inject\1\javax.inject-1.jar;D:\development\eclipse\mars.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;D:\development\eclipse\mars.m2\repository\com\sun\jersey\contribs\jersey-guice\1.19\jersey-guice-1.19.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\jackson\module\jackson-module-jaxb-annotations\2.7.8\jackson-module-jaxb-annotations-2.7.8.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\jackson\jaxrs\jackson-jaxrs-json-provider\2.7.8\jackson-jaxrs-json-provider-2.7.8.jar;D:\development\eclipse\mars.m2\repository\com\fasterxml\jackson\jaxrs\jackson-jaxrs-base\2.7.8\jackson-jaxrs-base-2.7.8.jar;D:\development\eclipse\mars.m2\repository\org\apache\hadoop\hadoop-hdfs-client\3.1.1\hadoop-hdfs-client-3.1.1.jar;D:\development\eclipse\mars.m2\repository\com\squareup\okhttp\okhttp\2.7.5\okhttp-2.7.5.jar;D:\development\eclipse\mars.m2\repository\com\squareup\okio\okio\1.6.0\okio-1.6.0.jar;D:\development\eclipse\mars.m2\repository\com\google\inject\extensions\guice-servlet\4.0\guice-servlet-4.0.jar;D:\development\eclipse\mars.m2\repository\io\netty\netty\3.10.5.Final\netty-3.10.5.Final.jar;D:\development\eclipse\mars.m2\repository\com\google\guava\guava\27.0.1-jre\guava-27.0.1-jre.jar;D:\development\eclipse\mars.m2\repository\com\google\guava\failureaccess\1.0.1\failureaccess-1.0.1.jar;D:\development\eclipse\mars.m2\repository\com\google\guava\listenablefuture\9999.0-empty-to-avoid-conflict-with-guava\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\development\eclipse\mars.m2\repository\org\checkerframework\checker-qual\2.5.2\checker-qual-2.5.2.jar;D:\development\eclipse\mars.m2\repository\com\google\errorprone\error_prone_annotations\2.2.0\error_prone_annotations-2.2.0.jar;D:\development\eclipse\mars.m2\repository\com\google\j2objc\j2objc-annotations\1.1\j2objc-annotations-1.1.jar;D:\development\eclipse\mars.m2\repository\org\codehaus\mojo\animal-sniffer-annotations\1.17\animal-sniffer-annotations-1.17.jar;D:\development\eclipse\mars.m2\repository\org\slf4j\slf4j-log4j12\1.7.22\slf4j-log4j12-1.7.22.jar;C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 15.0.4\lib\idea_rt.jar” com.intellij.rt.execution.application.AppMain org.apache.parquet.tools.Main column-index –debug C:\Users\masayuki\Downloads\phone-book-writer.parquet row group 0: column index for column id: Boudary order: ASCENDING null count min max
page-0 0 17 199

offset index for column id: offset compressed size first row index page-0 4 876 0

column index for column name: Boudary order: ASCENDING null count min max
page-0 2 alice thing2

offset index for column name: offset compressed size first row index page-0 880 876 0

column index for column location.lon: Boudary order: ASCENDING null count min max
page-0 40 100.0 199.0

offset index for column location.lon: offset compressed size first row index page-0 1756 589 0

column index for column location.lat: Boudary order: ASCENDING null count min max
page-0 73 200.0 398.0

offset index for column location.lat: offset compressed size first row index page-0 2345 325 0

column index for column phoneNumbers.phone.number: Boudary order: ASCENDING null count min max
page-0 3 100 5555555555

offset index for column phoneNumbers.phone.number: offset compressed size first row index page-0 2670 887 0

column index for column phoneNumbers.phone.kind: Boudary order: ASCENDING null count min max
page-0 5 cell mobile

offset index for column phoneNumbers.phone.kind: offset compressed size first row index page-0 3596 44 0