Running a web server with Quart

[cldtech]

Hi, this is not exactly a bug i assume but nothing in the rather short documentation or from online searches could help me. I am trying to build an android app that runs a Quart app in background with Chaquopy and a web view to display it full screen. The problem is i can't find a way to run Chaquopy without blocking the main thread which is kind of pointless, no matter what i try there is an error!

Here's why i can't figure it out:

• Running a web server in Chaquopy is a blocking long-running operation which blocks the android thread execution therefore the UI even if the UI is in a thread. So Chaquopy cannot run in the main android thread.
• Some part of Quart (seems to be asyncio) requires it to run in the main thread of the main interpreter.

So is it simply impossible to run a python web server and a webview in the same android app?

Running Chaquopy in the activity like such blocks the main thread and the UI (webview) stops working:

class MainActivity : AppCompatActivity() {

    private val tag: String = "MainActivity" // Define a tag for your logs
    private lateinit var webView: WebView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        Log.i(tag, "Starting server");
        if (! Python.isStarted()) {
            Python.start(AndroidPlatform(this))
        }
        // Run the Python server
        try {
            val py = Python.getInstance()
            py.getModule("app").callAttr("run")
        } catch (e: Exception) {
            Log.e(tag, "Error starting Python server: ${e.message}")
        }
        
        webView = findViewById(R.id.webview)
        webView.webViewClient = WebViewClient()
        webView.settings.javaScriptEnabled = true
        webView.loadUrl("http://localhost:5000")
    }

}

I tried running Chaquopy in an android thread as such:

class MainActivity : AppCompatActivity() {

    private val tag: String = "MainActivity" // Define a tag for your logs
    private lateinit var webView: WebView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        Log.i(tag, "Starting server");
        if (! Python.isStarted()) {
            Python.start(AndroidPlatform(this))
        }
        // Run the Python server in a background thread
        Thread {
            try {
                val py = Python.getInstance()
                py.getModule("app").callAttr("run")
            } catch (e: Exception) {
                Log.e(tag, "Error starting Python server: ${e.message}")
            }
        }.start()
        webView = findViewById(R.id.webview)
        webView.webViewClient = WebViewClient()
        webView.settings.javaScriptEnabled = true
        webView.loadUrl("http://localhost:5000")
    }

}

Which give me this error from the activity:

Error starting Python server: RuntimeError: set_wakeup_fd only works in main thread of the main interpreter

I tried running the server in a Python thread like this:

import threading
from quart import Quart

app = Quart(__name__)


@app.route('/')
async def hello():
    return 'hello'

def run():
    # Create a new thread
    thread = threading.Thread(target=app.run, args=("1",))
    # Start the thread
    thread.start()
    # Optionally, wait for the thread to finish
    thread.join()

Which give me this error in logcat through python.stderr:

Exception in thread Thread-1 (run):
Traceback (most recent call last):
File "stdlib/asyncio/unix_events.py", line 105, in add_signal_handler
ValueError: set_wakeup_fd only works in main thread of the main interpreter

I tried running Chaquopy in a coroutine:

class MainActivity : AppCompatActivity() {

    private val tag: String = "MainActivity" // Define a tag for your logs
    private lateinit var webView: WebView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        Log.i(tag, "Starting server");
        if (! Python.isStarted()) {
            Python.start(AndroidPlatform(this))
        }
        // Launch a coroutine to run Python code
        CoroutineScope(Dispatchers.IO).launch {
            runPythonCode()
        }
        // Run the Python server in a background thread
        Thread {

        }.start()
        webView = findViewById(R.id.webview)
        webView.webViewClient = WebViewClient()
        webView.settings.javaScriptEnabled = true
        webView.loadUrl("http://localhost:5000")
    }

    private suspend fun runPythonCode() {
        // Switch to the main thread to update UI if needed
        withContext(Dispatchers.Main) {
            try {
                val py = Python.getInstance()
                py.getModule("app").callAttr("run")
            } catch (e: Exception) {
                Log.e(tag, "Error starting Python server: ${e.message}")
            }
        }
    }

}

Which does start the server but still blocks the main android thread and cause this error:

ANR in com.example.chatterbot (com.example.chatterbot/.MainActivity)
 PID: 12527
 Reason: Input dispatching timed out (7aa0a25 com.example.chatterbot/com.example.chatterbot.MainActivity (server) is not responding. Waited 10007ms for FocusEvent(hasFocus=true))
 Parent: com.example.chatterbot/.MainActivity
 ErrorId: f902fe3a-f5fa-40be-907d-fab77be9f0ca
 Frozen: s[false] g[false]
 Load: 26.58 / 26.37 / 26.93
------ Current CPU Core Info ------
 - offline : 
 - online : 0-7
  0           1           2           3           4           5           6           7
------------------------------------------------------------------------------------------------------------------
  scaling_cur_freq       2301000     2301000     2301000     2301000      501000      501000      501000      501000
  scaling_governor     schedutil   schedutil   schedutil   schedutil   schedutil   schedutil   schedutil   schedutil
  scaling_max_freq       2301000     2301000     2301000     2301000     1800000     1800000     1800000     1800000                                                                                  ------------------------------------------------------------------------------------------------------------------
----- Output from /proc/pressure/memory -----
some avg10=1.41 avg60=1.40 avg300=1.01 total=8666562438
full avg10=1.10 avg60=0.93 avg300=0.64 total=5116135829
----- End output from /proc/pressure/memory -----                                       
  CPU usage from 0ms to 12700ms later (2024-09-13 18:29:29.762 to 2024-09-13 18:29:42.462):
  52% 3492/system_server: 33% user + 19% kernel / faults: 57585 minor 80 major
  14% 157/kswapd0: 0% user + 14% kernel
  0.2% 1040/media.swcodec: 0% user + 0.1% kernel / faults: 27482 minor 79 major
  5.9% 12527/com.example.chatterbot: 3.4% user + 2.4% kernel / faults: 26634 minor 11 major
  4.9% 307/exe_cq/0: 0% user + 4.9% kernel
  0% 995/media.extractor: 0% user + 0% kernel / faults: 6019 minor 28 major
  3% 4187/com.sec.imsservice: 1.5% user + 1.4% kernel / faults: 8117 minor 179 major
  2.9% 3843/com.android.systemui: 1.8% user + 1.1% kernel / faults: 9067 minor 33 major
  2.6% 3805/com.android.phone: 1.6% user + 1% kernel / faults: 7118 minor 41 major
  2.5% 1023/android.hardware.sensors@2.0-service-mediatek: 0.7% user + 1.8% kernel / faults: 132 minor
  2.5% 25148/adbd: 0.8% user + 1.6% kernel / faults: 152 minor
  2.3% 3253/lmkd: 0.5% user + 1.8% kernel
  2.2% 5150/com.sec.android.sdhms: 1% user + 1.1% kernel / faults: 5288 minor 71 major
  2% 1699/main_thread: 0% user + 2% kernel
  1.8% 1700/hif_thread: 0% user + 1.8% kernel
  1.7% 26185/com.menny.android.anysoftkeyboard: 1% user + 0.7% kernel / faults: 5768 minor 73 major
  0% 1022/media.codec: 0% user + 0% kernel / faults: 3617 minor 18 major
  1.5% 481/logd: 0.3% user + 1.1% kernel / faults: 90 minor
  1.4% 559/android.system.suspend@1.0-service: 0% user + 1.4% kernel / faults: 201 minor
  1.4% 3846/com.android.networkstack.process: 0.6% user + 0.8% kernel / faults: 5223 minor 27 major
  1.4% 9013/kworker/u16:2-events_unbound: 0% user + 1.4% kernel
  1.4% 7357/process-tracker: 0.1% user + 1.2% kernel / faults: 66 minor
  1.3% 21881/kworker/u16:5-pvr_misr: 0% user + 1.3% kernel
  1.2% 4170/com.sec.sve: 0.5% user + 0.7% kernel / faults: 4977 minor 44 major
  1.2% 32560/com.google.android.webview:sandboxed_process0:org.chromium.content.app.SandboxedProcessService0:0: 0.9% user + 0.3% kernel / faults: 4390 minor 4 major
  1.1% 802/surfaceflinger: 0.5% user + 0.6% kernel / faults: 1328 minor 5 major
  1% 4206/com.android.se: 0.5% user + 0.4% kernel / faults: 4264 minor 33 major
  0.9% 1701/rx_thread: 0% user + 0.9% kernel
  0.9% 11542/com.google.android.gms: 0.3% user + 0.5% kernel / faults: 2885 minor 29 major
  0.8% 7899/logcat: 0.3% user + 0.5% kernel / faults: 45 minor
  0.7% 215/chre_kthread: 0% user + 0.7% kernel
  0.7% 327/ipi_cpu_dvfs_rt: 0% user + 0.7% kernel
  0.7% 12389/com.google.android.webview:sandboxed_process0:org.chromium.content.app.SandboxedProcessService0:0: 0.6% user + 0.1% kernel / faults: 1593 minor

I don't know what to try next and can't find much online, is running a webserver in Chaquopy whthout blocking the UI impossible?

[mhsmith]

set_wakeup_fd only works in main thread of the main interpreter

The first Google result for this error message indicates that it was fixed in Python 3.9. To change the Python version of your app, follow these instructions.

Alternatively, if you call Python.start on a different thread, then Python will consider that to be the main thread. It doesn't need to be the same as the main thread in Java.

[cldtech]

Of course i googled this, but it still doesn't work. I was already on Python 3.12, i tried 3.9 just to make sure but it does the same errors. But Putting Python.start in the thread did it, i just had to remove android:name="com.chaquo.python.android.PyApplication" from the manifest.
Thanks for this project by the way, it's much needed!

[cldtech]

I spoke too fast, it worked with an empty Quart app but as soon as i add a call to Kotlin from Python i get Error starting Python server: Can't create handler inside thread Thread[Thread-4,5,main] that has not called Looper.prepare() from MainActivity. So i added a call to Looper but that gives me Error starting Python server: Method addObserver must be called on the main thread even though the call to run the Quart app is in the thread where the Python interpreter is started.

Kotlin:

package com.example.chatterbot

import android.annotation.SuppressLint
import android.net.http.SslError
import android.os.Bundle
import android.os.Looper
import android.util.Log
import android.webkit.SslErrorHandler
import android.webkit.WebView
import android.webkit.WebViewClient
import androidx.appcompat.app.AppCompatActivity
import androidx.swiperefreshlayout.widget.SwipeRefreshLayout
import com.chaquo.python.Python
import com.chaquo.python.android.AndroidPlatform

class MainActivity : AppCompatActivity() {

    private val tag: String = "MainActivity" // Define a tag for your logs
    private var serverPort: Int? = null
    private lateinit var webView: WebView
    private lateinit var swipeRefreshLayout: SwipeRefreshLayout

    @SuppressLint("SetJavaScriptEnabled")
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        // Run the Python server in a background thread
        Thread {
            Looper.prepare()
            Log.i(tag, "Starting server")
            Log.i(tag, "IS STARTED")
            Log.i(tag, Python.isStarted().toString())
            if (! Python.isStarted()) {
                Python.start(AndroidPlatform(this))
            }
            try {
                val py = Python.getInstance()
                py.getModule("app").callAttr("run")
            } catch (e: Exception) {
                Log.e(tag, "Error starting Python server: ${e.message}")
            }
            Looper.loop()
        }.start()

        // Set webview
        Log.i(tag, "Starting webview")
        webView = findViewById(R.id.webview)
        webView.webViewClient = object : WebViewClient() {
            override fun onReceivedSslError(
                view: WebView?,
                handler: SslErrorHandler?,
                error: SslError?
            ) {
                // Ignore SSL certificate errors (not recommended for production)
                handler?.proceed()
            }
        }
        webView.settings.javaScriptEnabled = true
        webView.loadUrl("file:///android_asset/iframe/iframe.html")
        swipeRefreshLayout = findViewById(R.id.swipe_refresh_layout)
        swipeRefreshLayout.setOnRefreshListener {
            webView.reload() // Reload the current page
            swipeRefreshLayout.isRefreshing = false // Stop the refreshing animation
        }
    }

    fun closeLoader() {
        Log.i(tag, "Close loader")
        webView.evaluateJavascript("closeLoader();", null)
    }

}

Python:

from hypercorn.asyncio import serve
from hypercorn.config import Config
from quart import Quart, render_template
from com.example.chatterbot import MainActivity

activity = MainActivity() # <-- **The problem comes when i add this**

app = Quart(__name__)

config = Config()
config.bind = [f"0.0.0.0:8000"]
config.debug = True

@app.route('/')
async def index():
    return await render_template('index.html')

@app.after_serving
async def after_startup():
    # This code will run after the server is fully loaded and ready to receive requests
    print("Server ready")
    activity.closeLoader()

def run():
    app.run(host="0.0.0.0", port=8000, debug=True)

[mhsmith]

If you want help with an error message, you must always post the full stack trace.

[mhsmith]

activity = MainActivity() # <-- **The problem comes when i add this**

You never create an activity explicitly in Android. The system creates it, and calls methods on it. If you want to access it from Python code, just pass this as an argument when you call run, and store it somewhere on the Python side.

[cldtech]

I see, this line was given to me by both ChatGPT and Llama, i know these are not great sources it was a just a try since i didn't totally got the doc right and there's not much online yet for Chaquopy that i have seen. Anyhow thanks!
Another clarification about using the android activity in Python if i may. Are properties suppose to be accessible or is it only methods? I tried accessing a public variable of the activity with dot notation and i get an error.

Kotlin:

class MainActivity : AppCompatActivity() {

    private val tag: String = "MainActivity" // Define a tag for your logs
    private lateinit var webView: WebView
    private lateinit var swipeRefreshLayout: SwipeRefreshLayout
    var port: Int? = null

    @SuppressLint("SetJavaScriptEnabled")
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        //Get available port
        Log.i(tag, "Request available port")
        val socket = ServerSocket(0)
        port = socket.localPort
        socket.close()

        // Run the Python server in a background thread
        Thread {
            // Is that in background? Try in a bound service for lifecycle
            Looper.prepare()
            if (! Python.isStarted()) {
                Log.i(tag, "Starting Python")
                Python.start(AndroidPlatform(this))
            }
            try {
                Log.i(tag, "Running web app")
                val py = Python.getInstance()
                py.getModule("app").callAttr("run", this)
            } catch (e: Exception) {
                Log.e(tag, e.stackTraceToString())
                Log.e(tag, "Error starting Python server: ${e.message}")
            }
            Looper.loop()
        }.start()

    }

Python:

app = Quart(__name__)
...
def run(activity):
    logger.info('Running server')
    main_activity = activity
    app.run(host="0.0.0.0", port=main_activity.port, debug=True)

Error:

2024-09-19 15:50:34.301  4011-4140  MainActivity            com.example.chatterbot               E  com.chaquo.python.PyException: AttributeError: 'MainActivity' object has no attribute 'port'
                                                                                                    	at <python>.java.chaquopy.setup_object_class.JavaObject.__getattribute__(class.pxi:216)
                                                                                                    	at <python>.java.chaquopy.setup_object_class.JavaObject.__getattribute__(class.pxi:222)
                                                                                                    	at <python>.app.run(app.py:53)
                                                                                                    	at <python>.chaquopy_java.call(chaquopy_java.pyx:354)
                                                                                                    	at <python>.chaquopy_java.Java_com_chaquo_python_PyObject_callAttrThrowsNative(chaquopy_java.pyx:326)
                                                                                                    	at com.chaquo.python.PyObject.callAttrThrowsNative(Native Method)
                                                                                                    	at com.chaquo.python.PyObject.callAttrThrows(PyObject.java:232)
                                                                                                    	at com.chaquo.python.PyObject.callAttr(PyObject.java:221)
                                                                                                    	at com.example.chatterbot.MainActivity.onCreate$lambda$1(MainActivity.kt:82)
                                                                                                    	at com.example.chatterbot.MainActivity.$r8$lambda$2Cl2IeR6D4NJ8-N3MnHOqsemGRQ(Unknown Source:0)
                                                                                                    	at com.example.chatterbot.MainActivity$$ExternalSyntheticLambda3.run(D8$$SyntheticClass:0)
                                                                                                    	at java.lang.Thread.run(Thread.java:1012)
2024-09-19 15:50:34.301  4011-4140  MainActivity            com.example.chatterbot               E  Error starting Python server: AttributeError: 'MainActivity' object has no attribute 'port'

[mhsmith]

Are properties suppose to be accessible or is it only methods?

Java fields are accessible, but Kotlin "properties" must be accessed via their get and set methods as described here. I agree it would be a good idea to support the Kotlin-like syntax (#668).