Access graphics rendering on Android device from MCU via USB

Community Projects
1

I want to share my project idea and know want you think about it.

**I want to make use of Android device possibilities in MCU projects. But instead of writing specialized application for every project in Android Studio, I want to make an app which will provide android hardware related features to MCU based device via USB.** **So for any project I will have to write only MCU code and I will just access Android hardware resources like LCD, touchscreen, memory storage, various sensors, audio playback over USB.**

For example, we can create virtual frame buffer in Android app, and provide access to basic drawing functions over USB CDC to STM32. Functions like filling rectangle at the specified position, filling texture from image file, drawing circle, rotation, double buffering…

Further I want to use drawing functions provided over USB CDC with LVGL (see https://github.com/lvgl) on STM32. So for example, rectangle drawing function in LVGL would be replaced with rectangle drawing function which will actually send rectangle drawing command together with coordinates and color or texture image name to Android over USB. Also get touchscreen press data over USB.

I think 480Mbs USB (and even 12Mbs) should be just more than enough to run LVGL or use any hardware features of Android.

By this way we can have 2k AMOLED display with GPU accelerated drawing directly accessible from MCU.

What you think? And why such app is still not available? Or I did not found it maybe? If so, glad to know such project link.

2

Maybe the need to use a separate device to interact with an MCU is considered to be a disadvantage for some applications. Still, I would expect that there will be projects out there that do something like what you want just because it’s an interesting way to provide interactivity to “headless” devices.

There are various remote display protocols, and you can also take a higher level approach by exporting user interface elements instead of low-level graphics primitives. I expect that there are web frameworks that do both of these things. Running a web server on an embedded device is also a way to achieve a basic level of interactivity.

Sorry I can’t be more helpful with specific recommendations. It’s been a while since I’ve looked at user interface frameworks.