I'm getting the flow of using
asyncio in Python 3.5 but I haven't seen a description of what things I should be
awaiting and things I should not be or where it would be neglible. Do I just have to use my best judgement in terms of "this is an IO operation and thus should be
I'm getting the flow of using
By default all your code is synchronous. You can make it asynchronous defining functions with
async def and "calling" these functions with
await. A More correct question would be "When should I write asynchronous code instead of synchronous?". Answer is "When you can benefit from it". In cases when you work with I/O operations as you noted you will usually benefit:
# Synchronous way: download(url1) # takes 5 sec. download(url2) # takes 5 sec. # Total time: 10 sec. # Asynchronous way: await asyncio.gather( async_download(url1), # takes 5 sec. async_download(url2) # takes 5 sec. ) # Total time: only 5 sec. (+ little overhead for using asyncio)
Of course, if you created a function that uses asynchronous code, this function should be asynchronous too (should be defined as
async def). But any asynchronous function can freely use synchronous code. It makes no sense to cast synchronous code to asynchronous without some reason:
# extract_links(url) should be async because it uses async func async_download() inside async def extract_links(url): # async_download() was created async to get benefit of I/O html = await async_download(url) # parse() doesn't work with I/O, there's no sense to make it async links = parse(html) return links
One very important thing is that any long synchronous operation (> 50 ms, for example, it's hard to say exactly) will freeze all your asynchronous operations for that time:
async def extract_links(url): data = await download(url) links = parse(data) # if search_in_very_big_file() takes much time to process, # all your running async funcs (somewhere else in code) will be frozen # you need to avoid this situation links_found = search_in_very_big_file(links)
You can avoid it calling long running synchronous functions in separate process (and awaiting for result):
executor = ProcessPoolExecutor(2) async def extract_links(url): data = await download(url) links = parse(data) # Now your main process can handle another async functions while separate process running links_found = await loop.run_in_executor(executor, search_in_very_big_file, links)
One more example: when you need to use
requests in asyncio.
requests.get is just synchronous long running function, which you shouldn't call inside async code (again, to avoid freezing). But it's running long because of I/O, not because of long calculations. In that case, you can use
ThreadPoolExecutor instead of
ProcessPoolExecutor to avoid some multiprocessing overhead:
executor = ThreadPoolExecutor(2) async def download(url): response = await loop.run_in_executor(executor, requests.get, url) return response.text
You have already called
loop.close() before you ran that sample piece of code, on the global event loop:
>>> import asyncio >>> asyncio.get_event_loop().close() >>> asyncio.get_event_loop().is_closed() True >>> asyncio.get_event_loop().run_until_complete(asyncio.sleep(1)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/.../lib/python3.6/asyncio/base_events.py", line 443, in run_until_complete self._check_closed() File "/.../lib/python3.6/asyncio/base_events.py", line 357, in _check_closed raise RuntimeError('Event loop is closed') RuntimeError: Event loop is closed
You need to create a new loop:
loop = asyncio.new_event_loop()
You can set that as the new global loop with:
and then just use
Alternatively, just restart your Python interpreter, the first time you try to get the global event loop you get a fresh new one, unclosed.
As of Python 3.7, the process of creating, managing, then closing the loop (as well as a few other resources) is handled for you when use
asyncio.run(). It should be used instead of
loop.run_until_complete(), and there is no need any more to first get or set the loop.
When should one implement a service?
When you have work -- delivering value to the user -- that:
Needs some time to complete, perhaps longer than you have time for in the component wishing the work to be done, or
Is delivering that value under user control (e.g., music player, controlled by play/pause buttons in a UI), or
In rare cases, needs to be running continuously, as it delivers value continuously
there are quite a lot of them running and I doubt it's just a poor application design
Some are likely to be poor implementations, either due to technical misunderstandings, or other concerns (e.g., making marketing happy) trumping making users happy.
It was a background sound recorder and I was using it as Foreground service with notification since I wanted buttons to be able to control it (like music players do for example)
That is a reasonable use for a service, IMHO.
Intent listeners are quite a useful feature and using them as you know is usually enough for many use-cases (for example showing notifications)
I assume that by "Intent listeners" you mean manifest-registered
BroadcastReceivers. In that case, if the work to be done by the
BroadcastReceiver will take more than a millisecond, that work should be delegated to an
IntentService for completion.
onReceive() is called on the main application thread, and it is not safe for a manifest-registered
BroadcastReceiver to fork a bare thread, as the process could go away shortly after
onReceive() returns. However, in these cases, the service is usually short-lived (e.g., do some network I/O and disk I/O, then go away).
In the development documentation services are suggested for network communication and background calculations but I don't get why you should not just use AsyncTasks for that
AsyncTask is a fine solution for background work that is:
Requested by the UI (activity or fragment), and
Will take less than a second or so, and
For example, if you are downloading avatars to show in a
AsyncTask is probably a fine choice, whether you use them directly or use some image-fetching library that uses them internally.
Conversely, if the user buys an MP3 through your app, and you need to download that MP3 file, an
AsyncTask is not a good solution. That could easily take over a second. While the download is going on, the user could switch away from the app (e.g., press HOME). At that point, your process is eligible to be terminated... perhaps before your download is complete. Using an
IntentService to manage the download is a signal to the OS that you are really doing work here, adding value to the user, and so the process will be left alone for a little while.
Note that if the background work might take 15+ seconds,
WakefulBroadcastReceiver or my
WakefulIntentService is probably a good idea, so the device does not fall asleep while you are trying to wrap up this bit of work.
First, you should consider using loop.run_in_executor with a ProcessPoolExecutor if you plan to run python subprocesses from within the loop. As for your problem, you can use the event loop policy functions to set a new loop:
import asyncio from concurrent.futures import ProcessPoolExecutor async def sub_main(): print('Hello from subprocess') def sub_loop(): loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) loop.run_until_complete(sub_main()) async def start(executor): await asyncio.get_event_loop().run_in_executor(executor, sub_loop) if __name__ == '__main__': executor = ProcessPoolExecutor() asyncio.get_event_loop().run_until_complete(start(executor))
If txt is already equal to a jquery object, there is no need to use $(txt) as it's just extra processing to return the same thing.