Apache spark snappy error java.lang.UnsatisfiedLinkError - failed to map segment from shared object

When writing simple compressed parquet from spark -on linux- you may get the following error. There are multiple solutions to the problem.

Caused by: java.lang.IllegalArgumentException: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.2-1928b886-bc92-4a6a-959c-44fb3c2225fe-libsnappyjava.so: /tmp/snappy-1.1.2-1928b886-bc92-4a6a-959c-44fb3c2225fe-libsnappyjava.so: failed to map segment from shared object: Operation not permitted [org.apache.spark.launcher.OutputRedirector.redirect]
at org.apache.spark.io.SnappyCompressionCodec$.liftedTree1$1(CompressionCodec.scala:171) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.apache.spark.io.SnappyCompressionCodec$.org$apache$spark$io$SnappyCompressionCodec$$version$lzycompute(CompressionCodec.scala:168) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.apache.spark.io.SnappyCompressionCodec$.org$apache$spark$io$SnappyCompressionCodec$$version(CompressionCodec.scala:168) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.apache.spark.io.SnappyCompressionCodec.(CompressionCodec.scala:152) [org.apache.spark.launcher.OutputRedirector.redirect]
... 18 more [org.apache.spark.launcher.OutputRedirector.redirect]
Caused by: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.2-1928b886-bc92-4a6a-959c-44fb3c2225fe-libsnappyjava.so: /tmp/snappy-1.1.2-1928b886-bc92-4a6a-959c-44fb3c2225fe-libsnappyjava.so: failed to map segment from shared object: Operation not permitted [org.apache.spark.launcher.OutputRedirector.redirect]
at java.lang.ClassLoader$NativeLibrary.load(Native Method) [org.apache.spark.launcher.OutputRedirector.redirect]
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965) [org.apache.spark.launcher.OutputRedirector.redirect]
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890) [org.apache.spark.launcher.OutputRedirector.redirect]
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851) [org.apache.spark.launcher.OutputRedirector.redirect]
at java.lang.Runtime.load0(Runtime.java:795) [org.apache.spark.launcher.OutputRedirector.redirect]
at java.lang.System.load(System.java:1062) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:174) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.xerial.snappy.Snappy.(Snappy.java:47) [org.apache.spark.launcher.OutputRedirector.redirect]
at org.apache.spark.io.SnappyCompressionCodec$.liftedTree1$1(CompressionCodec.scala:169) [org.apache.spark.launcher.OutputRedirector.redirect]

...

The issue is with linking the snappy libraries. If you check the logs, you will find something like this
Caused by: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.2-1928b886-bc92-4a6a-959c-44fb3c2225fe-libsnappyjava.so: /tmp/snappy-1.1.2-1928b886-bc92-4a6a-959c-44fb3c2225fe-libsnappyjava.so: failed to map segment from shared object: Operation not permitted

The main issue here is execute permission on the /tmp folder which is not granted by default on linux. One solution is to remount the /tmp with execute permission.

sudo mount /tmp -o remount,exec

Another solution would be to pass another directory to spark -that has execute permission on it- through JVM arguments.

-Dorg.xerial.snappy.tempdir=~/tmp