Capsules are CapsNet

capsules

A new type of neural network 

Capsules network a brand new type of artificial neural networks that is superior to CNN is here to stay. Prepare for at least one detailed post about them in near future.

Where do I find a working implementation?

Please refer to this implementation of CapsNet in Keras. This repository provides links to almost all known current implementations of freshly brewed Capsules network.

The bright future

My intuition indicates that Capsules will substitute CNN in a near future due to there in-variance to image position transformation.

Resources

Papers

Dynamic Routing Between Capsules

Matrix capsules with EM routing

Implementations

CapsNet-Keras

CapsuleNet on MNIST

Blogs

Capsule Networks Explained

What is a CapsNet or Capsule Network?

YouTube

Capsule Networks: An Improvement to Convolutional Networks (Siraj Raval)

News

GOOGLE’S AI WIZARD UNVEILS A NEW TWIST ON NEURAL NETWORKS

Advertisements

Deep Learning virus. Are you infected?

Taken by surprise

It’s hardly possible to find a single person that hasn’t heard about Deep Learning virus epidemics. The size of affected population is quite significant and the infection is spreading faster than was ever imagined. Who would have thought that such an esoteric virus could spread so rapidly. The main question is how the authorities missed this case completely until the point when it is very little that can be done to fight this strong and capable, not to say a kind of intelligent adversary.

What went wrong?

The virus origin dates back to nineteen sixties and seventies when there was reported that a couple of scientists were affected by Deep Learning virus which then had no such a name and was known as Perceptron. However, it was thought that timely treatment of a newly discovered XOR antibiotic cured it completely though sporadic eruptions of it were reported also in mid eighties.

The things started to change suddenly in 2012. Although, a few years before this there were a number of cases when people from speech recognition community were affected by Deep Learning virus. What happened in 2012 though was more significant since for the first time it was reported that vision, and more precisely, object classification functionality, was strongly affected by it. 

Today, we are witnessing a new wave of this infection and it’s unsettling to see that this virus has grown to become such a beast. We all know now that almost all human senses is affected by it be it vision, speech generation and recognition, hearing, cognitive functions such a primitive sentiment, you name it. It is unclear if sense of taste and smell are in danger but we cannot be overoptimistic in this regard.

Virus characteristics

At first, it was thought that somehow the virus only targets certain predisposed members of the population such as scientists and engineers like Geoffrey Hinton, Yann LeCun, Yoshua Bengio and others. It turned out that we were completely wrong in this assumption and the virus is much smarter and flexible than we thought possible. Now large fractions of population be it doctors, artists, entrepreneurs such as Elon Musk, and even renowned physicist such as Max Tegmark are deeply affected by this Deep Learning epidemics.

It is mutating

Throughout the years, researchers were able to uncover a number of mutations of Deep learning virus and we now know about Auto Encoders, Convolutional and Recurrent spices of it. Each day the most authoritative remedy journal Arxiv reports on newer cases of mutations that gives us little hope that the treatment will keep pace with the virus evolution. There are rumors that a new and unseen kind of mutation such as Deep Reinforcement Learning are even more dangerous not to mention generative adversarial networks and who knows what else to come.

On a side note, it is at least, a little bit reassuring to know that one of the mutations called Theano was eradicated by MILA and we hope that other will follow too.

Transmission

Apparently, the virus is transmuted via digital means of Internet publications, open-source, such as MXNet and usage of large corporations products such as Keras, TenserFlow, PyTorch. It is most probable that participation in conferences, such as NIPS and others can put you in immediate danger of being affected. So ask yourself before attending them if the risk worth it.

A new hope

Even though the virus is strong and unrelenting,  we place our hopes in development of new antibiotics, such as Numenat’s HTM or neuromorphic drugs, such as Neurogrid.

So be cautious and take all measures to fight the virus and hope that human intelligence will beat this sneaky, powerful, smart and flexibly entity that somehow learns to outsmart us each time we think we’ve found a cure. 

Take care.

 

A Digest of Deep Learning Pearls

All you need is time and GPU

Try to allocate time for these thought provoking Deep Learning papers. Part of them with try it yourself implementation at GitHub.

1. Try it yourself at home or anywhere at all (with GPU)

Transformer more than meet the eye!
– A novel approach to language understanding from Google Brain(via David Ha)
It is a very interesting solution for an old linguistic/ syntactic challenge (anaphora) with Deep Learning. More detailed explanation of anaphora resolution.
– Based on “Attention is all you needpaper

2. Learning To Remember Rare Events

An interesting approach to introduce memory module into various types of Deep Learning architectures to provide them with life long learning.

3. One Model To Learn Them All

A unified Deep Learning model that is capable of being applied to inputs from various modalities. It is a one step closer toward general DL architectures.

4. Meet Fashion-MNIST

Finally, it is time to ditch MNIST in favor of Fashion-MNIST

Which is better from a number of aspects. Which one? Find yourself.

**Note:

If you haven’t noticed the one thing in common to all of these items except for one is
Łukasz Kaiser researcher from Google Brain.

 Java Code Geeks

NLP is Natural Language Processing

Get ready for a real NLP

I am back to blogging and have a motivation to post a number of posts (or at least one) on the subject of Natural Language Processing. Upcoming posts also will contain information on recurrent neural networks such as LSTM. So stay tuned.

For now, check this out

If you are into Natural Language Processing (NLP) then you may find links below useful.

Papers

1. Attention Is All You Need paper in arxiv.

 

 

No winter but AI global warming

formula

Name things for what they are

Is Deep Learning rage simply a bubble or is this time it here for a long time to stay. As researchers proposed first let’s change the Deep Learning title into the more humble and exact Multilayered Network for Functions Approximation. Now it sounds more practical and there is no sign of hype. Then check to what fields those networks were applied and see if it is diverse and if the algorithms used are universally applicable. Check the number of articles published that have a real essence within them. If you’ve got ‘yes’ as an answer to those questions then it feels like finally those approaches are really usefull.

What’s next?

This post will be updated in a near future. Meanwhile check the posts by Carlos E. Perez from IntuitionMachine.com that writes extensively on the subject and do not forget to check his ‘The Deep Learning Playbook

Wind of Deep Change

Welcome to the world of Machine and Deep Learning

Following my transition to another continent in near future I’ll be able to focus more on Machine and Deep Learning being a technical editor at renowned Machine Learning Mastery site authored by Dr. Jason Brownlee. It means you can expect more posts on machine learning to come especially on LSTM and recurrent neural networks.

What is it like to be a technical editor?

Throughout my career I’ve been a SW test engineer and SW developer but in parallel I’ve been busy helping to edit books as Jumping Into C++ by Alex Allain and other projects, such as Kindle Optimizer Chrome extension. So becoming a technical editor in machine learning field is just a logical step to make. Actually technical editor is a bit like a QA engineer and a developer at once since you have to understand how Python code is working to make that LSTM to be able to predict time series values and to be a test engineer to make the content and the code to be as good as it can be. In addition, there is a kind of freedom that regular tester or developer do not possess which is to suggest changes to the author which may be meaningful and influential. Most importantly, technical editor deals with the raw content of a future article, a blog post or a chapter from the book that millions of people may read and it provides you with the understanding of the responsibility that you bear on your shoulders. The corrections that you make may influence readers and make their experience pleasant or not.

Why machine or deep learning after all?

Technical editing as testing or programming is a universal position since it can be successfully applied to various topics in those fields, but machine learning has the proper ingredients of math, programming and future potential that makes it very attractive.

Stay tuned as John Sonmez says

So if you follow this blog stay around the corner to be up to date with the current progress in Deep Learning field and if you care check this public Deep Learning for All group at Facebook where I share latest and in my view greatest news coming from Deep Learning fruitful field.

 

 

OpenCV installation on Linux and Windows

ubuntu_opencv-1

How hard is to install OpenCV?

This was the question that I asked myself lately when I needed to use OpenCV for a project. I thought it must be simpler on Ubuntu than on Windows. But I was wrong. The goal of this tutorial is to provide working guidelines for OpenCV installation. I’ll cover installation instructions for OpenCV with following configurations:

Windows 7/ 10 

  • OpenCV 3.x.x with Python 2.7
  • OpenCV 3.x.x with Python 3.5

Ubuntu 16.04 

  • OpenCV 3.x.x with Python 3.5

Installation on Windows 7/ 10

OpenCV 3.x.x with Python 2.7 on Windows 32 bit

To have all the dependencies that are related to Python it is useful to install Anaconda.

  • Install Anaconda 2 for Python 2.7 (32 or 64 bit)
  • Install Anaconda 3 for Python 3.5 (32 or 64 bit)
  • Now we can install OpenCV by using pre-built libraries by downloading them from here.
  • For the sake of this tutorial I used OpenCV version 3.2.0
    • opencv-3.2.0-vc14.exe
  • After you’ve installed downloaded OpenCV version there is a need to move cv2.pyd file to a Python installation library.

Look for the cv2.pyd at the opencv installation folder

C:\Users\You\Downloads\opencv\build\python\2.7\x64\cv2.pyd

And move the cv2.pyd file to Python 2.7 installation folder

C:\Users\You\Anaconda2\Lib\site-packages\cv2.pyd

python_2.7.png

Example application

  • To test that opencv installed correctly
  • Open command line and run python. Then type the commands below to figure out what is the current opencv version.
C:\Users\You>python
Python 2.7.13 |Anaconda 4.3.0 (32-bit)| (default, Dec 19 2016, 13:36:02) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import cv2
>>> print(cv2.__version__)
3.2.0
>>>

OpenCV 3.x.x with Python 3.5 using Wheel on Windows 7 64 bit

  • There is no library for Python 3.5 support in OpenCV out of the box that is why we can use  unofficial Windows binaries for Python extension packages from here to be able to use it.

Note: I downloaded this one because I have Windows 7 64 bit

  • opencv_python-3.2.0-cp35-cp35m-win_amd64.whl

Pay attention that 3.2.0 means opencv version i.e. opencv-3.2.0

cp35 means Python version i.e. Python 3.5

  • After you downloaded this file open the command line and open the directory this file located in. For example, let’s say it was downloaded to Downloads folder.
  • Change current folder to Downloads 
C:\>cd C:\Users\You\Downloads
  • Install wheel with pip install command
C:\Users\You\Downloads>pip install opencv_python-3.2.0-cp35-cp35m-win_amd64.whl
Processing c:\users\andrei\downloads\opencv_python-3.2.0-cp35-cp35m-win_amd64.whl
Installing collected packages: opencv-python
Successfully installed opencv-python-3.2.0
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

C:\Users\You\Downloads>
  • Pay attention that you saw this line ‘Successfully installed opencv-python-3.2.0’

Example application

  • To test that opencv installed correctly
  • Open command line and run python. Then type the commands below to figure out what is the current opencv version.
C:\Users\You\Downloads>python
Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print(cv2.__version__)
3.2.0
>>>

Additional resources

Installation on Ubuntu 16.04

To install OpenCV on Ubuntu follow the steps in the guides below. The first one is the best and it worked for me.

  • Simply run this command for basic opencv3 installation.
conda install -c menpo opencv3
  • If Anaconda is not installed then run this one to install it.
sudo apt-get install python-opencv

Additional resources

What’s next?

Now that you have a working OpenCV you may watch this nice tutorial by Siraj Raval that is funny and hands on with OpenCV. It will teach you How to do Object Detection with OpenCV. It will also teach you that there is a need to run a code at least once before filming a YouTube video.

In addition if you are interested in object detection with OpenCV then definitely look at Satya Mallick tutorial on the subject.

 Java Code Geeks