wtorek, 24 lutego 2015

Hadoop How to

How to disable "speculative execution"?
configuration.setBoolean("mapred.map.tasks.speculative.execution", false);
configuration.setBoolean("mapred.reduce.tasks.speculative.execution", false);
How to set notification url?
configuration.set("job.end.notification.url", request.getNotificationUrl());
How to attach all required jars to the job? configuration.set("tmpjars", StringUtils.join(request.getJars(), ","));

piątek, 20 lutego 2015

Sending POST Request with command line


AngularJs JSONP

http://www.codeproject.com/Articles/42641/JSON-to-JSONP-Bypass-Same-Origin-Policy Chrome error:
Uncaught SyntaxError: Unexpected token :
Firefox error:
SyntaxError: missing ; before statement
XMLHttpRequest cannot load http://hadoop05:12010/sqoop/version?id=1. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.

czwartek, 19 lutego 2015

> Ambiguous method overloading for method java.io.File#.
  Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
   [class java.lang.String]
   [class java.net.URI]

This error happend when you not set AIR_HOME and FLEX_HOME properly.
$FLEX_HOME/bin/adt -certificate -validityPeriod 25 -cn SelfSigned 1024-RSA sampleCert.pfx


/opt/adobe-air-sdk/bin/adt -package -target apk -storetype pkcs12 -keystore sampleCert.pfx DemoWeb.apk DemoWeb.xml demo_web.swf

środa, 11 lutego 2015

Dockerfile - how to run bash script without using external files

The usual way of smuggling external scripts to docker image builds is by adding them from directory where Dockerfile is located (ADD command). But this cause a need to maintain external resources which is not so handy, especially when scripts are rather short. The workaround is to echo all the commands and pass them to sh:
RUN echo -e "su -c \"echo 'N' | hdfs namenode -format\" hdfs \n\
su -c \"hdfs datanode 2>&1 > /var/log/hadoop-hdfs/hadoop-hdfs-datanode.log\" hdfs& \n\
su -c \"hdfs namenode 2>&1 > /var/log/hadoop-hdfs/hadoop-hdfs-namenode.log\" hdfs& \n\
sleep 5 \n\
/usr/lib/hadoop/libexec/init-hdfs.sh \n\
killall java \n\
" | sh

wtorek, 3 lutego 2015

GIMP - how to export image to base64

That was somewhat extremely surprising to me that GIMP does not provide Base64 exporter out of the box (but I am not the GIMP expert - maybe there is ;)). Nonetheless, as it seems to me a quite straightforward to implement such feature, I faced the challenge. After short googling and refreshing deep buried Python knowledge, it became simple, amazingly simple in fact. Here you are some details in three simple steps: 0. Open "Python Console" (don't be afraid, it does not bite you, or choke either ;)
Go to "Filters" menu -> "Python-Fu". The python console should appear.
1. Take an image
img = gimp.image_list()[0]  // assuming you'd got just one image opened
2. Read all bytes
bytes = open(img.filename, "rb").read()
3. Encode bytes with 'base64' algorithm

And for "one-liner" fanboys, we can compact it into one line:

open(gimp.image_list()[0].filename, "rb").read().encode('base64')
bytes.encode('base64') puts some new-line characters, alternatively you can use the following:
import base64
base64.b64encode(open(gimp.image_list()[0].filename, "rb").read())
Reference: Python Scripting Official Doc

poniedziałek, 2 lutego 2015

How to obtain Eclipse absolute installation path programmatically

The easiest way is to use org.eclipse.core.runtime.Platform class and its static getInstallLocation method.
Location installLocation = Platform.getInstallLocation();
A Location interface is defined in OSGi specification and basically represents an java.net.URL object. Additionally it allows creating nested locations and mechanism for locking locations. The default and the only one shipped implementation is BasicLocation, but it is internal class and just pure implementation (no other fancy features ;) To obtain path as a String user the following code:
String absolutePath = Platform.getInstallLocation().getURL().getFile();
Alternatively you can read the content
InputStream is = Platform.getInstallLocation().getURL().getContent();