Hadoop How to

How to disable "speculative execution"?

configuration.setBoolean("mapred.map.tasks.speculative.execution", false);
configuration.setBoolean("mapred.reduce.tasks.speculative.execution", false);

How to set notification url?

configuration.set("job.end.notification.url", request.getNotificationUrl());

How to attach all required jars to the job? configuration.set("tmpjars", StringUtils.join(request.getJars(), ","));

Sending POST Request with command line

http://superuser.com/questions/149329/what-is-the-curl-command-line-syntax-to-do-a-post-request

AngularJs JSONP

http://www.codeproject.com/Articles/42641/JSON-to-JSONP-Bypass-Same-Origin-Policy Chrome error:

Uncaught SyntaxError: Unexpected token :

Firefox error:

SyntaxError: missing ; before statement

XMLHttpRequest cannot load http://hadoop05:12010/sqoop/version?id=1. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.

> Ambiguous method overloading for method java.io.File#.
  Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
   [class java.lang.String]
   [class java.net.URI]

This error happend when you not set AIR_HOME and FLEX_HOME properly.

https://helpx.adobe.com/air/kb/archived-air-sdk-version.html
http://labs.adobe.com/downloads/air.html

$FLEX_HOME/bin/adt -certificate -validityPeriod 25 -cn SelfSigned 1024-RSA sampleCert.pfx

    .starling.samples.android.DemoWeb
    0.0.1
    HelloWorld
    
        demo_web.swf
    
    mobileDevice
  
  
          <!--[CDATA[
              
                  
                  
                  
                  
                  
                  
                  
                      
                          
                        
                        
                          
                      
                  
              
          ]]>
      
    

/opt/adobe-air-sdk/bin/adt -package -target apk -storetype pkcs12 -keystore sampleCert.pfx DemoWeb.apk DemoWeb.xml demo_web.swf

Dockerfile - how to run bash script without using external files

The usual way of smuggling external scripts to docker image builds is by adding them from directory where Dockerfile is located (ADD command). But this cause a need to maintain external resources which is not so handy, especially when scripts are rather short. The workaround is to echo all the commands and pass them to sh:

RUN echo -e "su -c \"echo 'N' | hdfs namenode -format\" hdfs \n\
su -c \"hdfs datanode 2>&1 > /var/log/hadoop-hdfs/hadoop-hdfs-datanode.log\" hdfs& \n\
su -c \"hdfs namenode 2>&1 > /var/log/hadoop-hdfs/hadoop-hdfs-namenode.log\" hdfs& \n\
sleep 5 \n\
/usr/lib/hadoop/libexec/init-hdfs.sh \n\
killall java \n\
" | sh

GIMP - how to export image to base64

That was somewhat extremely surprising to me that GIMP does not provide Base64 exporter out of the box (but I am not the GIMP expert - maybe there is ;)). Nonetheless, as it seems to me a quite straightforward to implement such feature, I faced the challenge. After short googling and refreshing deep buried Python knowledge, it became simple, amazingly simple in fact. Here you are some details in three simple steps: 0. Open "Python Console" (don't be afraid, it does not bite you, or choke either ;)

Go to "Filters" menu -> "Python-Fu". The python console should appear.

1. Take an image

img = gimp.image_list()[0]  // assuming you'd got just one image opened

2. Read all bytes

bytes = open(img.filename, "rb").read()

3. Encode bytes with 'base64' algorithm

bytes.encode('base64')

Done.

And for "one-liner" fanboys, we can compact it into one line:

open(gimp.image_list()[0].filename, "rb").read().encode('base64')

bytes.encode('base64') puts some new-line characters, alternatively you can use the following:

import base64
base64.b64encode(open(gimp.image_list()[0].filename, "rb").read())

Reference: Python Scripting Official Doc

How to obtain Eclipse absolute installation path programmatically

The easiest way is to use org.eclipse.core.runtime.Platform class and its static getInstallLocation method.

Location installLocation = Platform.getInstallLocation();

A Location interface is defined in OSGi specification and basically represents an java.net.URL object. Additionally it allows creating nested locations and mechanism for locking locations. The default and the only one shipped implementation is BasicLocation, but it is internal class and just pure implementation (no other fancy features ;) To obtain path as a String user the following code:

String absolutePath = Platform.getInstallLocation().getURL().getFile();

Alternatively you can read the content

InputStream is = Platform.getInstallLocation().getURL().getContent();