Browsed by
Month: August 2015

GSoC 2015 – The Processing Foundation

GSoC 2015 – The Processing Foundation

GSoC 2015 has come to an end. I was lucky to have such an intelligent mentor, Andres Colubri.

I worked on maintaining the Android mode of Processing. My GSoC project was around the following main goals :

  • Update the Android Mode to work with the updated processing base code.
  • Move PApplet from Activity to Fragment so that it can be embedded inside other apps.
  • Create a video library for the Android Mode of Processing.

Let’s start with the first sub task. It was all about laying down the ground work for the development of the later stages. Processing base went ahead and processing-android was left as it is. I fixed the build system and the Android mode itself so that sketches were at least being compiled without any problem.

Second task was to move PApplet from Activity to Fragment. The biggest benefit for this change is that users can now embed a processing sketch in their own app. Since, Fragment has its own life cycle, the developer doesn’t have to worry about any of the intricacies at all. The user just needs to create an object of the PApplet (which is a child class of Fragment class) and add this Fragment to his/her Activity.

For a demo of this feature head over to my Github – ProcessingAndroidDemo

The third and the main subtask of my project was to create a video library for Processing for Android. The video library can take input from either the device’s camera or a video file present locally on the device. Processing has it’s own image format called PImage over which we can do all the processing operations. This library presents two classes Capture and Movie, both of them are the child classes of the PImage class. For example, the Capture class represents the camera frame at any given instant of time. Since, Capture extends PImage, we can do stuffs like using shaders, applying this PImage texture over a 3D shape etc.

The video library gives an awesome performance because of no GPU to CPU data transfer at any stage. Everything is handled inside the GPU itself. Although, I’ve implemented loadPixels() which gives user the access to raw pixels but is pretty slow because the data has to be transferred from GPU to CPU.

For a demo of the video library, have a look here – Processing Video for Android library – Demo