Youtube Is Becoming Interesting

For all these years its been hard to find high quality material on YouTube, mostly because of all the cat and/or would be funny videos. But great, high quality content is slowly being produced and gaining attention.

The following video showcases the some of the engineering behind the aluminum can. Its impressive I tell you. Go watch!.

https://www.youtube.com/watch?v=hUhisi2FBuw

The Great War Youtube Channel is Interessting

Originally, I created this blog to post about interesting stuff. It turns out that most interesting stuff I know is usually well known and already linked from millions of places on the internet. Also I ended-up posting only about programming related stuff.

Well for once I get to post about something interesting that no enough people know about. The Great War YouTube channel is making a free gigantic documentary on World War 1. Every week they post a 10 minutes episode which describes the events that took place 100 years ago the same week. Because the pace of the documentary is tied to the pace of the events as they took place, it gives a perspective that a 2 hour documentary cannot.

This is the most captivating historical documentary I have ever watched! Its free! So have a look, its amazing!

Getting more output out of C++ Boost Unittest Framework

I spent a more than 15 minutes to figure that out so I will note this here. I was trying to get as much output as possible from my Boost C++ unittests, and somehow the output file was smaller if I used the sink options.

This is what gave me the biggest result file in my tests:
-r detailed -l all >results.txt 2>&1

This is the documentation for the unittest program arguments:
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/utf/user-guide/runtime-config/reference.html

Playing with PlayN: Kick-Ass Crossplatform Game Development

When looking at its main sites, its hard believe that its been more than one year since Google revealed PlayN, then known as ForPlay, to the general public. The amazing technology allows to develop a game in java and target almost any platform: HTML5, Android, ios, Flash, Java (and XNA someday?).

Disclamer: I'm not a Java developer. I knew nothing of Maven, Ant, GWT, war files, JAVA_HOME, JAVA_JRE, just yesterday. It took me many hours to make the sample projects work.

First I had to install the JRE and eclipse, by default it downloaded the 32 bits JRE which did not work with the 64 bit eclipse I had downloaded, I also installed Git and got the samples. I tried to follow the eclipse guide but I encountered many unexpected errors. Most of which I found solutions by googling. A link leading to a maven related .jar file was broken and I had to download it manually and put it in my .m2 folder in my MyDocuments folder. Don't ask me why it puts all the jar files there, I was surprised when I could not find them in my projects.

Starting the samples in eclipse did not work, I had to use the ant method, I found the location of the ant eclipse comes with, added it to my PATH and ran the ant commands in the guide. An other annoyance was that the java source paths in eclipse were not set correctly so I could not follow declarations in the source.

I also installed Tomcat and realized how easy it is to deploy the generated HTML5 version.

So the getting started doc and maven project might need some fixing. Then I realized that the version of playN in the sample is not the latest release, also, the latest version has many improvements listed in the release notes. I set myself to find out how to generate a PlayN 1.4 project.

I tried to generate the project with eclipse but it just does not work, there is a problem with Maven and many discussion threads on the web suggested generating the project with the stand alone version of maven (not the Eclipse plugin). So I install the JDK (needed for Maven). Installed Maven.

The command to generate the project was not too hard to guess:
mvn archetype:generate -DarchetypeGroupId=com.googlecode.playn -DarchetypeArtifactId=playn-archetype -DarchetypeVersion=1.4

Then I imported the project in eclipse. Unlike the sample projects, the dependencies between projects are not set correctly for the skeleton projects. RunAs->Maven Test did not work at first on any of the projects, I had to do RunAs->Maven Install on the core project, then RunAs->Maven Install on the HTML5 and Java projects (the versions I want to work with). After doing these RunAs->Maven Test did work on both the HTML5 and Java.

I'm still confused about the structure of the projects. There is a project for each platform and then there is an appname-core and appname project which both contains the same source files, but one of them is not considered as a Java project. I'm not sure how I'm is supposed to work, where to put the unit tests, what are all those Maven commands, ect.

So, to come back to the main subject, its hard to believe that the main site is in that state. With that amazing technology, show-cased by on of the most popular mobile games (Angry Birds), one would think that there would be a community out there and that the documentation would be abundant, and that it would be easy to get started. Also I could not find a link the google developers page. I had to seach for playN on the site, while way less interesting project are advertised in the game section. What I understand is that Maven thing is supposed to make things work automagically, but by not working it makes the whole process complicated. Probably, bearded seasoned Java/Maven/Eclipse/GWT developers out there have no problem setting their own project how they like and are too lazy to update the docs and sample projects. That's too bad, think of all the fun kids could have making their own games and deploying them on the web. If not for the problems related to the maven projects, PlayN would be a great introduction to programming and game making. The engine itself looks easy to program, and the Java and HTML5 are easy deploy.

Interesting links:

• Google I/O 2011: Kick-Ass Game Programming with Google Web Toolkit
• Google Code Link
• Google Developers Link

Encrypting a file in Windows

Recently I’ve been thinking about backing up some of my files into the cloud. I would want to encrypt my files before sending them into the cloud just in case my account gets compromised. I started looking for a free/open source command line solution for file encryption. Some of the best open source solution I had found only offered source code download and I didn’t feel like installing a compiler on my windows machine just to try them. This gave me the motivation to find way to use something already installed into windows to do the encryption. I found that powershell was already installed on that machine and that it had access to .Net. That’s great.

On windows, too often, the quickest and easiest way to do something is to download a third .party software to do it, even if you already have the code somewhere in the operating system. Windows does not come with a compiler, it does not come with compiled applications to use all its features, it is not really empowering its users by it self. At least now it has powershell.

So, I made a small powershell script that creates an AES encrypted copy of a file. In the end I will probably not use it because I remembered that 7zip which is already installed on my machine supports encryption, is able to sync a directory in compressed archives and has a command line interface.

Anyways here is the script:

param(
    [string]$keyfile=$(throw "Parameter keyfile required."),
    [string]$infile=$(throw "Parameter infile required."),
    [string]$outfile=$(throw "Parameter outfile required."),
    [switch]$decrypt
    )
# Encrypt Usage:
# powershell -ExecutionPolicy Unrestricted ./aesfile.ps1 -keyfile key.txt -infile data.txt -outfile data.txt.aes
# Decrypt Usage:
# powershell -ExecutionPolicy Unrestricted ./aesfile.ps1 -keyfile key.txt -infile data.txt.aes -outfile data.dec.txt -decrypt

# Read the key and the iv
# each is on one line and has its byte written in decimal ascii separated by ":"
[byte[]]$key = (Get-Content $keyfile)[0]| foreach {($_ -split ":")}
[byte[]]$iv = (Get-Content $keyfile)[1]| foreach {($_ -split ":")}
[Console]::Error.WriteLine("key=$key")
[Console]::Error.WriteLine("IV=$iv")
[Console]::Error.WriteLine("infile=$infile")
[Console]::Error.WriteLine("outdile=$outfile")
[Console]::Error.WriteLine("decrypt=$decrypt")

# Setup FileStreams
[System.IO.Stream]$instream = new-object System.IO.FileStream $infile, ([System.IO.FileMode]::Open)
[System.IO.Stream]$outstream = new-object System.IO.FileStream $outfile, ([System.IO.FileMode]::Create)

# Setup Encryption
$AesCSP = New-Object System.Security.Cryptography.AesCryptoServiceProvider
$AesCSP.Key = $key
$AesCSP.IV = $IV
if(!$decrypt) {$cryptor=$AesCSP.CreateEncryptor()}
else{$cryptor=$AesCSP.CreateDecryptor()}
$cstream = new-Object Security.Cryptography.CryptoStream $instream,$cryptor,"Read"

# Number of bytes to read with each chunks from the stream.
$BLOCK_BYTE_SIZE = $AesCSP.BlockSize/8
$NB_BLOCKS_READ = 100
$CHUNK_BYTE_SIZE = $BLOCK_BYTE_SIZE * $NB_BLOCKS_READ

#$cstream.CopyTo($outstream, $CHUNK_BYTE_SIZE) Only in .Net 4.0 :(
[byte[]]$buffer = new-object byte[] $CHUNK_BYTE_SIZE

while($byte_read=$cstream.Read($buffer, 0, $buffer.Length))
{
    $outstream.Write($buffer, 0, $byte_read)
}

$cstream.Close()
$instream.Close()
$outstream.Close()

Here is a sample key file:

247:74:89:63:153:30:170:243:11:247:227:187:2:177:244:17:154:9:150:12:155:253:168:19:224:241:254:148:213:50:147:184
129:15:216:20:169:11:167:32:75:57:109:137:15:14:92:212

The first line is the key(32 bytes), the second line is an initialization vector(16 bytes).
Bytes are written in decimal and separated by ‘:’.

Notes:
    I lost a lot of time trying to support reading binary input on stdin and never found a solution. For something that's called “powershell” I expected better support for stdin. The existing commands/functions only support reading text from the keyboard and the hidden $input.'<>4__this' can only read lines of text. I wanted to use my script with | and > instead of having to open the files myself.

F#, python, piping generators/iterators/IOs

A few weeks ago, I was watching this excellent introduction to F#, a programming language greatly inspired by other ML like languages.

http://channel9.msdn.com/pdc2008/TL11/

The pipeline operator |> is demonstrated in the video. It is argued that using this operator allows programmers to organize their code easily in a natural way.

This is meaningful for me because I've been avoiding a functional approach when using python because I would fear it would affect the readability of the code.

Look at at the following example, it starts with a few ordinary python examples, and then I added my "python pipe" response to F#.

# My Maths
def fA(x):
 print "In fA"
 return x+1

def fB(x):
 return x+2

def fC(x):
 print "In fC"
 return x+3

myInput = [1,2,3]

ugly = ( fA(i) for i in ( i+2 for i in ( fC(i) for i in myInput)))
print "About to process."
ugly = list(ugly) # execution starts here
print "ugly: %s" % ugly

# Lamda makes generator expressions reusable with different inputs
genA = lambda x : ( fA(i) for i in x)
genB = lambda x : ( i+2 for i in x)
genC = lambda x : ( fC(i) for i in x)

lessUgly = genA(genB(genC(myInput)))
lessUgly = list(lessUgly)
print "lessUgly: %s" % lessUgly

#or
genCi = genC(myInput)
genBi =genB(genCi)
genAi = genA(genBi)
# At least that way, generators appear in the "right" order

result = list(genAi)
print "result: %s" % result

#In newer versions of python map returns an iterator
# in older version take a look at imap in itertools
# (I'm currently python 2.6)

mapResult = map(fA, map(fB, map(fC, myInput)))
print "mapResult: %s" % mapResult

#or 

mapC = map(fC, myInput)
mapB = map(fB, mapC)
mapA = map(fA, mapB)

mapResult = list(mapA)
print "mapResult: %s" % mapResult


# Now with a pipe!

class Pipe:
 "Handles flow."
 
 def __init__ (self, source):
  self.lastIter = iter(source)
 
 def __iter__(self):
  return self.lastIter
 
 def __or__(self, right):
  try:
   it = iter(right)
  except:
   return NotImplemented
  self.lastIter = it
  return self


p = Pipe(myInput)
# Notice the generators appear in the order they are executed
p | ( fC(i) for i in p) | ( i+2 for i in p) | ( fA(i) for i in p)
result = list(p)
print "Piperesult: %s" % result

#or

p = Pipe(myInput)

p | ( fC(i) for i in p)
p | ( i+2 for i in p)
p | ( fA(i) for i in p)
result = list(p)
print "Piperesult: %s" % result

# or with map
p = Pipe(myInput)

p | map(fC, p)
p | map(fB, p)
p | map(fA, p)
result = list(p)
print "Piperesult: %s" % result

If you did watch the whole F# video, you might recall that at the end, threading is added just by adding a few keywords and symbols. I'm sure they are also many easy way to add threading to this approach in python. Maybe with a threaded map? tmap(numberOfThreads=4, inputBuffer=5, outputBuffer=5, myFunction, myInput). Or if the Pipe class would handle functions directly instead of iterators, it could decide on a threading strategy by itself?

Anyways that's it for today.