EDIT: 01/12/2021 previous answer (find it at the bottom) didn’t age well therefore I added a combination of possible solutions that may satisfy those who still look on how to co-use asyncio and Celery Lets quickly break up the use cases first (more in-depth analysis here: asyncio and coroutines vs task queues): If the task … Read more
To add a function to an already running event loop you can use: asyncio.ensure_future(my_coro()) In my case I was using multithreading (threading) alongside asyncio and wanted to add a task to the event loop that was already running. For anyone else in the same situation, be sure to explicitly state the event loop (as one … Read more
praw relies on the requests library, which is synchronous meaning that the code is blocking. This can cause your bot to freeze if the blocking code takes too long to execute. To get around this, a separate thread can be created that handles the blocking code. Below is an example of this. Note how blocking_function … Read more
Are there any practical differences between Options 1 and 2? No. Option 2 looks nicer and might be marginally more efficient, but their net effect is the same. I know run_until_complete will run until the future has completed, so since Option 1 is looping in a specific order I suppose it could behave differently if … Read more
I think this is most likely an aiohttp bug. Specifically, I found this issue on their github: https://github.com/aio-libs/aiohttp/issues/4324 I realise this doesn’t necessarily help you, but maybe you can switch back and stop banging your head against a wall. Your code is fine!
If I understand you well, you want to limit the number of simultaneous requests? There is a object inside asyncio named Semaphore, it works like an asynchronous RLock. semaphore = asyncio.Semaphore(50) #… async def limit_wrap(url): async with semaphore: # do what you want #… results = asyncio.gather([limit_wrap(url) for url in urls]) updated Suppose I make … Read more
The asyncio.run() documentation says: This function cannot be called when another asyncio event loop is running in the same thread. In your case, jupyter (IPython ≥ 7.0) is already running an event loop: You can now use async/await at the top level in the IPython terminal and in the notebook, it should — in most of the … Read more
How do I terminate such long running CPU-bound computations within a method? The approach you tried doesn’t work because the futures returned by ProcessPoolExecutor are not cancellable. Although asyncio’s run_in_executor tries to propagate the cancellation, it is simply ignored by Future.cancel once the task starts executing. There is no fundamental reason for that. Unlike threads, … Read more
The missing of public method like loop.run_once() is intentional. Not every supported event loop has a method to iterate one step. Often underlying API has methods for creating event loop and running it forever but emulating single step may be very ineffective. If you really need it you may implement single-step iteration easy: import asyncio … Read more
asyncio.as_completed() takes an iterable of coroutines or futures and returns an iterable of futures in the order that the input futures complete. Normally, you’d loop over its result and await the members from inside an async function… import asyncio async def first(): await asyncio.sleep(5) return ‘first’ async def second(): await asyncio.sleep(1) return ‘second’ async def … Read more