Deep Learning virus. Are you infected?

Taken by surprise

It’s hardly possible to find a single person that hasn’t heard about Deep Learning virus epidemics. The size of affected population is quite significant and the infection is spreading faster than was ever imagined. Who would have thought that such an esoteric virus could spread so rapidly. The main question is how the authorities missed this case completely until the point when it is very little that can be done to fight this strong and capable, not to say a kind of intelligent adversary.

What went wrong?

The virus origin dates back to nineteen sixties and seventies when there was reported that a couple of scientists were affected by Deep Learning virus which then had no such a name and was known as Perceptron. However, it was thought that timely treatment of a newly discovered XOR antibiotic cured it completely though sporadic eruptions of it were reported also in mid eighties.

The things started to change suddenly in 2012. Although, a few years before this there were a number of cases when people from speech recognition community were affected by Deep Learning virus. What happened in 2012 though was more significant since for the first time it was reported that vision, and more precisely, object classification functionality, was strongly affected by it. 

Today, we are witnessing a new wave of this infection and it’s unsettling to see that this virus has grown to become such a beast. We all know now that almost all human senses is affected by it be it vision, speech generation and recognition, hearing, cognitive functions such a primitive sentiment, you name it. It is unclear if sense of taste and smell are in danger but we cannot be overoptimistic in this regard.

Virus characteristics

At first, it was thought that somehow the virus only targets certain predisposed members of the population such as scientists and engineers like Geoffrey Hinton, Yann LeCun, Yoshua Bengio and others. It turned out that we were completely wrong in this assumption and the virus is much smarter and flexible than we thought possible. Now large fractions of population be it doctors, artists, entrepreneurs such as Elon Musk, and even renowned physicist such as Max Tegmark are deeply affected by this Deep Learning epidemics.

It is mutating

Throughout the years, researchers were able to uncover a number of mutations of Deep learning virus and we now know about Auto Encoders, Convolutional and Recurrent spices of it. Each day the most authoritative remedy journal Arxiv reports on newer cases of mutations that gives us little hope that the treatment will keep pace with the virus evolution. There are rumors that a new and unseen kind of mutation such as Deep Reinforcement Learning are even more dangerous not to mention generative adversarial networks and who knows what else to come.

On a side note, it is at least, a little bit reassuring to know that one of the mutations called Theano was eradicated by MILA and we hope that other will follow too.

Transmission

Apparently, the virus is transmuted via digital means of Internet publications, open-source, such as MXNet and usage of large corporations products such as Keras, TenserFlow, PyTorch. It is most probable that participation in conferences, such as NIPS and others can put you in immediate danger of being affected. So ask yourself before attending them if the risk worth it.

A new hope

Even though the virus is strong and unrelenting,  we place our hopes in development of new antibiotics, such as Numenat’s HTM or neuromorphic drugs, such as Neurogrid.

So be cautious and take all measures to fight the virus and hope that human intelligence will beat this sneaky, powerful, smart and flexibly entity that somehow learns to outsmart us each time we think we’ve found a cure. 

Take care.

 

Advertisements

Kaitai Web IDE on Windows and Linux

kaitai

A Guide to Kaitai Web IDE

This post is a continuation of my previous post about Kaitai Struct DSL language for description of binary data formats. This post will describe how to download Kaitai Web IDE and run it locally as a web application and  also how to build and run it.

Prerequisites

If you are interested to run Kaitai Web IDE locally or take part in its development then there is a need to install some additional software, such as

  • Anaconda 2/3 that will help install
    • Git
    • Python 2/3
    • Node.js (to be able to build locally)

This guide will explain how to do it on Windows and Linux(Ubuntu). 

In addition the following versions will be used

  • Anaconda with Python 2.7
  • Git 2.14.1
  • Python 2.7
  • Node.js 6.11.0

Anaconda for the rescue

Anaconda is a package manager for various libraries be it Python, Node.js etc. We’ll use it to install all the dependencies we need for Kaitai Web IDE.

1. Download Anaconda from official site

conda.png

For the sake of this tutorial you can chose either Python 2.7 or 3.6 version.

2. Install Anaconda as any application.

Look for newly installed programs and run Anaconda Prompt (Windows)

anaconda.png

3. Open command line or Anaconda prompt and install Git (taken from here)

conda install -c conda-forge git

4. Approve if asked to install any additional packages.

5. Test that Git was installed correctly

C:\Users>git --version
git version 2.14.1.windows.1

6. Clone or download Kaitai Web IDE stable release from GitHub repository.

git clone https://github.com/kaitai-io/ide-kaitai-io.github.io

7. You’ll see output resembling this one

C:\Users>git clone https://g
ithub.com/kaitai-io/ide-kaitai-io.github.io
Cloning into 'ide-kaitai-io.github.io'...
remote: Counting objects: 3003, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 3003 (delta 0), reused 1 (delta 0), pack-reused 2995
Receiving objects: 100% (3003/3003), 5.98 MiB | 5.11 MiB/s, done.
Resolving deltas: 100% (2118/2118), done.

To run Kaitai Web IDE as is 

8. Open the folder where you placed Kaitai Web IDE (in my case

C:\Users\ide-kaitai-io.github.io

)

9. Run following command to lunch web server that will host locally Kaitai Web IDE app

python -mSimpleHTTPServer

10. Go to http://127.0.0.1:8000/

Note: pay attention that instead of port 8000 you may need to use any other such as 8888 if 8000 is used by some application if you see the error below

socket.error: [Errno 10013] An attempt was made to access a socket in a way forb
idden by its access permissions

10 (in case of error). To fix this execute previous command with a port as an argument

python -mSimpleHTTPServer 8888

Go to http://127.0.0.1:8888/

As you can see in the screenshot below I had to run on port 8888 and Kaitai Web IDE is up and running!

8888.png

Build it yourself

If you want to build Kaitai Web IDE yourself then install additional software with Anaconda.

The attempt below failed but the one after it passed. I preserve it for the sake of an argument (it can be done easily on Linux)

11. Install Node.js from here

conda install -c conda-forge nodejs 

12. Approve if asked to install any additional packages.

And there was an error on Windows (why is that?)

CondaError: WindowsError(206, 'The filename or extension is too long')

12 (once again). Ok. Then let’s download and install Node.js from official site.

node.png

13. Open command prompt and test Node.js version

C:\Users\iscamc>node --version
v6.11.3

We are almost on the finish line. To build Kaitai Web IDE

14. Clone or download Kaitai Web IDE stable release from GitHub repository.

git clone --recursive https://github.com/kaitai-io/kaitai_struct_webide

You’ll see output like this

C:\Users>git clone --recursi
ve https://github.com/kaitai-io/kaitai_struct_webide
Cloning into 'kaitai_struct_webide'...
remote: Counting objects: 5175, done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 5175 (delta 6), reused 12 (delta 5), pack-reused 5153
Receiving objects: 100% (5175/5175), 9.63 MiB | 4.49 MiB/s, done.
Resolving deltas: 100% (3852/3852), done.
Submodule 'formats' (https://github.com/kaitai-io/kaitai_struct_formats/) regist
ered for path 'formats'
Cloning into 'C:/Users/kaitai_struct_webide/formats'...
remote: Counting objects: 1576, done.
remote: Compressing objects: 100% (58/58), done.
remote: Total 1576 (delta 37), reused 63 (delta 23), pack-reused 1495
Receiving objects: 100% (1576/1576), 431.29 KiB | 2.82 MiB/s, done.
Resolving deltas: 100% (888/888), done.
Submodule path 'formats': checked out 'a3643b677daccfd323f7d9ace998292c9ee51811'

15. Open the folder where you placed Kaitai Web IDE (in my case 

C:\Users\ide-kaitai-io.github.io

)

16. Run following command to install all the JavaScript and TypeScript dependencies with 

npm install

17. Now  compile and run the Web IDE itself

python serve.py --compile

Note: pay attention that instead of port 8000 you may need to use any other such as 8888 if 8000 is used by some application if you see the error below

socket.error: [Errno 10013] An attempt was made to access a socket in a way forb
idden by its access permissions

To fix this error open in any text editor of your choice serve.py file that resides inside Kaitai folder in my case in C:\Users\kaitai_struct_webide and replace PORT=8000 with, say PORT=8888  on line 15.

line_15.png

17 (in case of error). Rerun command below

python serve.py --compile

When everything worked nice you’ll see something like this

C:\Users\kaitai_struct_webide>python serve.py --compile
Starting typescript compiler...
Please use 127.0.0.1:8888 on Windows (using localhost makes 1sec delay)
Press Ctrl+C to exit.
12:57:28 AM - Compilation complete. Watching for file changes.

18. Go to http://127.0.0.1:8888/

Congratulations you’ve build Kaitai Web IDE!

congrats.png

 Java Code Geeks

Web Code Geeks

Reverse Engineering with Kaitai Struct

Reverse engineering the easy way

Imagine you have some kind of 3rd party data storage that you need to understand how to work with and the only thing you have is a detailed description of the protocol using the device. The only problem is that there is no source code available that can make this process easy to accomplish. And what is left is to implement manually this protocol while having lots of trial and error iterations. Next time in similar occasion repeat this difficult process once again. But no worries, there is one tool that comes in handy in situations like this when there is a file or a stream that you want to parse and you want to be able to do it fast.  

Meet Kaitai Struct

First, here comes an official description of Kaitai Struct

Kaitai Struct is a domain-specific language (DSL) that is designed with one particular task in mind: dealing with arbitrary binary formats.

Parsing binary formats is hard, and that’s a reason for that: such formats were designed to be machine-readable, not human-readable. Even when one’s working with a clean, well-documented format, there are multiple pitfalls that await the developer: endianness issues, in-memory structure alignment, variable size structures, conditional fields, repetitions, fields that depend on other fields previously read, etc, etc, to name a few.

Kaitai Struct tries to isolate the developer from all these details and allow to focus on the things that matter: the data structure itself, not particular ways to read or write it.

Features

  • Kaitai is supported on Linux and Windows (not sure about Mac).
  • So far, Kaitai supports generating parsers in following languages
    • C++/STL
    • C#
    • Java
    • JavaScript
    • Perl
    • PHP
    • Python
    • Ruby
  • If you want you are welcome to add one more language to the list

How to use this Kaitai?

In short, to use Katai 

  • You use declarative syntax to describe a data source you want to be able to parse, such as file system or image format or whatever you like, in ksy file. 
  • Then using Kaitai Web IDE or Katai Struct compiler you generate a code in one of the relevant supported languages, such as Java, C#, C++ etc.
  • That’s it. Now use the code to get full access to your data source.

Kaitai REPL (Read–Eval–Print Loop

repl.png

To get a feeling what Kaitai is capable of you can start from playing with Kaitai REPL which has a number of examples showcasing what can be achieved with it, such as parsing doom.wad package files format.

Katai Web IDE

2017_09_17_23_16_07_Kaitai_Web_IDE.jpg

If you think you are ready to start applying Kaitai to real problems then jump into Katai Web IDE which is very nice and easy to use. You can upload there your data source and start writing a description of how the data source is organized. 

This official wiki page will show you the main features or Web IDE.

Kaitai Compiler stand alone 

It is possible to use Katai compiler in a stand alone mode via command line interface of your choice be it on Linux, Windows etc. How to do it is described here.

Resources

mikhail.png

 Java Code Geeks

A Digest of Deep Learning Pearls

All you need is time and GPU

Try to allocate time for these thought provoking Deep Learning papers. Part of them with try it yourself implementation at GitHub.

1. Try it yourself at home or anywhere at all (with GPU)

Transformer more than meet the eye!
– A novel approach to language understanding from Google Brain(via David Ha)
It is a very interesting solution for an old linguistic/ syntactic challenge (anaphora) with Deep Learning. More detailed explanation of anaphora resolution.
– Based on “Attention is all you needpaper

2. Learning To Remember Rare Events

An interesting approach to introduce memory module into various types of Deep Learning architectures to provide them with life long learning.

3. One Model To Learn Them All

A unified Deep Learning model that is capable of being applied to inputs from various modalities. It is a one step closer toward general DL architectures.

4. Meet Fashion-MNIST

Finally, it is time to ditch MNIST in favor of Fashion-MNIST

Which is better from a number of aspects. Which one? Find yourself.

**Note:

If you haven’t noticed the one thing in common to all of these items except for one is
Łukasz Kaiser researcher from Google Brain.

 Java Code Geeks

NLP is Natural Language Processing

Get ready for a real NLP

I am back to blogging and have a motivation to post a number of posts (or at least one) on the subject of Natural Language Processing. Upcoming posts also will contain information on recurrent neural networks such as LSTM. So stay tuned.

For now, check this out

If you are into Natural Language Processing (NLP) then you may find links below useful.

Papers

1. Attention Is All You Need paper in arxiv.

 

 

Deep Le… Space, Do You Copy?

600px-AS17-Flag_shots

There are other things too

In the middle of Deep Learning rush we forget that there are other things on this planet and off it that are fascinating. That’s right, I want to share with you the best materials I saw so far on Moon exploration that are highly recommended.

Books From Apollo Participants 

There are quite a few books written about US space program. But there are few that are really good. I’ve chanced to read some of them and below follow the best ones in my opinion.

The Last Man On The Moon Book

last1

The Last Man on the Moon: Astronaut Eugene Cernan and America’s Race in Space

This book is very special and it is a memoir by Gene Cernan the commander of Apollo 17. He was literally the last person to walk on the moon.

ProsThere is a special atmosphere in this book. The descriptions are so vivid and colorful. Gene Cernan was deeply touched by lunar visits since he was there twice on Apollo 10 and then Apollo 17. It is available on Kindle.

                                               Cons. It finished so fast. (No photos in the book)

The Last Man On The Moon Movie

lastmanmoon.jpg

There is also a movie named the same which may be found for free on the internet or bought here. Here is the trailer.

Two Sides of the Moon: Our Story of the Cold War Space Race

2side

Two Sides of the Moon: Our Story of the Cold War Space Race

This book combines recollections by Apollo 15 commander David Scott and his contemporary Alexi Leonov who was the first man to walk in space.

Pros. Very interesting book because of complementing accounts provided by both distinguished persons. Available in Kindle format.

Cons. Not a single photo.

From The Other Side

failureFailure Is Not an Option: Mission Control From Mercury to Apollo 13 and Beyond

The book below provides very different account of the matters described in the books above. It is written by Gene Kranz the Flight throughout entire US space program starting from Mercury and ending in Shuttle era.

Pros. The more technical book than astronauts accounts. Available on Kindle.

Cons. No photos again.

Documentaries that cannot be missed

  1. EARTHRISE: The First Lunar Voyage – documentary about Apollo 10.
  2. Apollo 13 Documentary 1958 – as it was portrayed by NASA.
  3. Apollo 15 Remembered 40 Years Later – documentary for Apollo 15 featuring Neil Armstrong and others.
  4. In The Shadow of The Moon – british documentary with interesting stuff.
  5. The Last Man On The Moon – documentary featuring Gene Cernan.
  6. Failure Is Not An Option A Flight Control History of NASA – documentary featuring flight controllers and Gene Kranz.
  7. Moon Machines – tools that made lunar program possible and people behind them.
  8. From The Earth To The Moon – a series produced by Tom Hanks. 

Not because they are easy, but because they are hard!

This post can’t be finished without the full inspirational to say the least speech by John F. Kennedy. It is incomparable to the current president of the US. It is a giant  speech for a president and a giant gap between then and now. 

potus.png

President Kennedy’s Speech at Rice University

No winter but AI global warming

formula

Name things for what they are

Is Deep Learning rage simply a bubble or is this time it here for a long time to stay. As researchers proposed first let’s change the Deep Learning title into the more humble and exact Multilayered Network for Functions Approximation. Now it sounds more practical and there is no sign of hype. Then check to what fields those networks were applied and see if it is diverse and if the algorithms used are universally applicable. Check the number of articles published that have a real essence within them. If you’ve got ‘yes’ as an answer to those questions then it feels like finally those approaches are really usefull.

What’s next?

This post will be updated in a near future. Meanwhile check the posts by Carlos E. Perez from IntuitionMachine.com that writes extensively on the subject and do not forget to check his ‘The Deep Learning Playbook