Browsed by
Tag: android

Create touch events programmatically in Android

Create touch events programmatically in Android

Creating a touch event in Android is pretty easy. Although, is the easy way the best? Let’s find out!
The simplest way is to use the “input” binary which you can use to create touch events all over the screen (via the shell user). The command to create a simple tap event is as simple as – adb shell input tap x y.

However, there’s a latency of more than 1 second while executing this command. If we’re making some automation related app, we definitely wouldn’t want this.

Another approach is to write values directly to the device node file available inside /dev for input. This approach has slightly lesser latency, but file writing operations are anyways expensive. On top of that, some devices/OEMs might have different locations/names for the device nodes which makes this approach a nightmare to implement.

Can we do better both in terms of consistency and latency? Definitely!

Luckily, there’s a class called InputManager in Android which has some interesting methods, although the methods of our interest are either private or hidden.

Now’s when Java’s Reflection API comes to the rescue. We’ll follow a series of hacky reflection API calls to get our input injection working.

First of all, we need the instance of InputManager class. For that, we’ll just invoke the getInstance method via reflection.

Next, we need to make obtain method of MotionEvent class accessible. Then, we will get the reflection reference of injectInputEvent method.

We’re all set now, and just need to write the code to actually pass the touch events to the Android system. The code involves creating a MotionEvent object, and calling the injectInputEvent via reflection.

Another awesome thing with this approach is that, the framework will itself figure out if the current event will be a normal tap, swipe or a long press.

The full implementation of this approach is present in RemoteDroid – https://github.com/omerjerk/RemoteDroid/blob/master/app/src/main/java/in/omerjerk/remotedroid/app/EventInput.java.

Where can you create touch events?

Of course, a random user (every app is run by its own user) can’t just create touch events on anywhere on the screen just like that. A particular app can create touch events on those views only which are owned by the same app.

But there’s a catch, shell user (uid 200) can create touch events all over the screen.

Now the question becomes, how would you start your app so that the app’s code is executed by the shell user. Either figure this out yourself or search my blog for something close to that. ūüėČ

Take screenshot programmatically without root in Android

Take screenshot programmatically without root in Android

Here’s another post related to Android hacking.

One can find a lot of ways over the internet which will tell you how take the screenshot of your own app, and that’s like pretty easy too. That’s also allowed by the Android framework. Note that here I’m not talking about the screenshots you take (as a user) by pressing Power + Volume down keys (that screenshot is taken by the SystemUI which has extra privileges). Let’s talk about the case if you want to take screenshot programatically from your own app/service and want to cover the whole screen.

According to Android security, if you were to take the screenshot of the screen programmatically, the only views visible will be the ones created by your own app. It won’t contain the views created by any other app.

So, I present a way to take screenshots of other views from your own app programmatically. The trick here is to use android’s MediaProjection API. Also, do note that this sets the minimum API level to be 21.

What we’ll do is to render the Android’s display contents on a Surface using the MediaProjection API. But then the toughest part is to get the contents of this Surface and create a proper Bitmap out of that. There’s no direct way of doing that.

Over the past few days, I tried different ways and failed –

  1. Create the Surface from¬†the encoder, then attach a decoder to the encoder output, and create a Bitmap from the decoder’s raw output. (Didn’t work and I wasn’t surprised)
  2. Create an OpenGL texture, create a Surface using that texture, then whatever is drawn on the Surface will automatically get passed to the OpenGL texture, and then call glReadPixels() to get raw pixel data to create the Bitmap. (Should’ve worked but for some reason, I was just getting a green colored image)
  3. Then I tried Android’s ImageReader API and that finally worked out.

Enough of the bullshit, let’s start with some code.

First of all, we need to initialize the object of the ImageReader class.

Now, we need to pass this surface to the MediaProjection API. Please go through this demo code to learn how to create the object of MediaProjection class.

Implement the ImageReader.OnImageAvailableListener in your class and do the following –

Also, make sure you have the following permission in your manifest.

In case you’re too lazy and when everything setup for you, here’s a small library I created – https://github.com/omerjerk/Screenshotter

Execute Java code as a root user in Android

Execute Java code as a root user in Android

You can find a lot of ways on StackOverflow about executing shell commands as root in Android, and it goes as follows :

What if you want to execute some Java code which uses Android APIs as a root user ?

This is not as straightforward as executing a shell command with root.

The idea is to compile the Java class with a static main method just like a normal apk is compiled.

First of all, we need to create a class which is to be executed as the root user. This class can be placed along with the normal classes inside your Android app. This should look like this (of course, the name of the class can be anything) :

Note that the package name of the above class is “in.omerjerk.remotedroid.app” which will be used to identify the above class later.

The hack is to start the above class just like how the Android’s framework starts the app when we click on the launcher icon. We use app_process binary to load our java class. The app_process is in class path and can be called by executing app_process32 inside the shell.
But, here’s the trick. We will start app_process as a root user, which will in turn load our class, again having root as the executing user.

The above paragraph boils down to the following command which is supposed to be executed in a rooted device.

In the above command, replace in.omerjerk.remotedroid.app and Main with your package name and class name respectively.

That’s all you need and your Java class will get executed as the root user.

How to install an app to /system partition ?

How to install an app to /system partition ?

Note : This article is for you¬†if you’re making an app for rooted android devices.

The normal behaviour being that the android’s package manager installs the apk file to /data partition. But for accessing hidden APIs and having extra privileges one may want the app to be installed in /system partition. For pre-kitkat devices the extra privileges folder is /system/app whereas for kitkat and post-kitkat devices the extra privileges folder is /system/priv-app.

The trick to install your app to /system partition is to first install it the normal way, then on the first run of the app, move the apk to /system/priv-app folder (/system/app for pre-kitkat devices). The following snippet of code makes your life easy to achieve this.

Use String.format() in your code to format the above commands as shown :

Execute the above formatted String after replacing the apk name, package name and the activity name to yours as shown :

I use libsuperuser to execute the SU commands in android. The way how to execute these commands is trivial and totally up to you.

Working example : https://github.com/omerjerk/RemoteDroid/blob/master/app/src/main/java/in/omerjerk/remotedroid/app/MainActivity.java#L31

Getting video stream from Android’s display

Getting video stream from Android’s display

This is something that has been tried to be achieved in various ways. What other people usually do is to take screenshots at regular intervals and stitch them together to make a video out of it. What I’m doing here is pretty different an much better than that approach.

And so I also present a way to capture video frames from Android’s default display and do further processing as one pleases.

I’ll broadly use two main APIs viz. MediaCodec¬†(added in API level 16) and DisplayManager¬†(added in API level 19). So this will limit our app to a minimum API level of 19 which is Kitkat. And further if you want to mirror the output of secure windows as well, then you’ll have to push your apk to /system/priv-app/ which will require having root access on your phone.

My logic is :

  • Create a video encoder.
  • Get an input Surface from the encoder using createInputSurface() method of the encoder object.
  • Pass this surface to DisplayManager so that the display manager will route it’s output to this surface.
  • Use the¬†dequeOutputBuffer()¬†method which will return you the H.264 encoded frames of your video.
  • And if you want to get raw video frames you can further pass these AVC encoded frames to a video decoder and get the raw frames. However as of now I’m not covering that in this blog post.

Let’s start with the code :

First we need to create an encoder, configure it and get an input surface out of it.

We will then pass the above created surface to createVirtualDisplay() method.

The DisplayManager will keep drawing the contents of the Android screen on our virtual display which in turn will feed the contents to video encoder through the surface.

The variable encodedData contains the AVC encoded frame and will be updated in every loop.
Now it’s upto you what you want to do with this. You can write it to a file. You can stream it over the network (Although that will require a lot more efforts).
You can further pass it to a video decoder to get raw frame and convert it to a bitmap or something. I’ve implemented the passing of video frames to video decoder in one of my projects here :
https://github.com/omerjerk/RemoteDroid/blob/master/app/src/main/java/in/omerjerk/remotedroid/app/ServerService.java#L285