我不懂Python的Asyncio

Recently I started looking into Python’s new asyncio module a bit more. The reason for this is that I needed to do something that works better with evented IO and I figured I might give the new hot thing in the Python world a try. Primarily what I learned from this exercise is that I it’s a much more complex system than I expected and I am now at the point where I am very confident that I do not know how to use it properly.

最近,我开始研究Python的新异步模块。 原因是我需要做一些事情来更好地处理事件IO,并且我想可以尝试一下Python世界中的新热点。 从这个练习中我主要了解到的是,它是一个比我预期的要复杂得多的系统,而现在我非常确信自己不知道如何正确使用它。

It’s not conceptionally hard to understand and borrows a lot from Twisted, but it has so many elements that play into it that I’m not sure any more how the individual bits and pieces are supposed to go together. Since I’m not clever enough to actually propose anything better I just figured I share my thoughts about what confuses me instead so that others might be able to use that in some capacity to understand it.

从概念上来说,它并不难理解,并且可以从Twisted中借鉴很多,但是它包含了太多的元素,因此我不确定各个片段如何组合在一起。 由于我不够聪明,无法提出更好的建议,所以我只是想分享我的想法,而不是让我感到困惑,以便其他人可以以某种方式使用它来理解它。

原始人 (The Primitives)

asyncio is supposed to implement asynchronous IO with the help of coroutines. Originally implemented as a library around the yield and yield from expressions it’s now a much more complex beast as the language evolved at the same time. So here is the current set of things that you need to know exist:

asyncio应该在协程的帮助下实现异步IO。 最初是作为围绕表达式的产量产量的库实现的,现在随着语言的同时发展,它已经变得更加复杂。 因此,这是您需要知道的当前事物集:

  • event loops
  • event loop policies
  • awaitables
  • coroutine functions
  • old style coroutine functions
  • coroutines
  • coroutine wrappers
  • generators
  • futures
  • concurrent futures
  • tasks
  • handles
  • executors
  • transports
  • protocols
  • 事件循环
  • 事件循环策略
  • 等待的
  • 协程功能
  • 旧式协程函数
  • 协程
  • 协程包装
  • 发电机
  • 期货
  • 并发期货
  • 任务
  • 处理
  • 执行者
  • 运输工具
  • 协议

In addition the language gained a few special methods that are new:

此外,该语言还获得了一些新的特殊方法:

  • __aenter__ and __aexit__ for asynchronous with blocks
  • __aiter__ and __anext__ for asynchronous iterators (async loops and async comprehensions). For extra fun that protocol already changed once. In 3.5 it returns an awaitable (a coroutine) in Python 3.6 it will return a newfangled async generator.
  • __await__ for custom awaitables
  • __aenter____aexit__用于块异步
  • __aiter____anext__用于异步迭代器(异步循环和异步理解)。 为了增加乐趣,该协议已经更改过一次。 在3.5中,它返回Python 3.6中的awaitable(协程),它将返回新的异步生成器。
  • __await__用于定制等待

That’s quite a bit to know and the documentation covers those parts. However here are some notes I made on some of those things to understand them better:

要知道很多,文档涵盖了那些部分。 但是,以下是我对其中一些内容所做的一些注释,目的是为了更好地理解它们:

事件循环 (Event Loops)

The event loop in asyncio is a bit different than you would expect from first look. On the surface it looks like each thread has one event loop but that’s not really how it works. Here is how I think this works:

asyncio中的事件循环与您初看时有所不同。 从表面上看,似乎每个线程都有一个事件循环,但实际上并非如此。 我认为这是这样的:

  • if you are the main thread an event loop is created when you call asyncio.get_event_loop()
  • if you are any other thread, a runtime error is raised from asyncio.get_event_loop()
  • You can at any point asyncio.set_event_loop() to bind an event loop with the current thread. Such an event loop can be created with the asyncio.new_event_loop() function.
  • Event loops can be used without being bound to the current thread.
  • asyncio.get_event_loop() returns the thread bound event loop, it does not return the currently running event loop.
  • 如果您是主线程,则在调用asyncio.get_event_loop()时会创建一个事件循环
  • 如果您是任何其他线程,则会从asyncio.get_event_loop()引发运行时错误
  • 您可以随时通过asyncio.set_event_loop()将事件循环与当前线程绑定。 可以使用asyncio.new_event_loop()函数创建这样的事件循环。
  • 可以使用事件循环而不必将其绑定到当前线程。
  • asyncio.get_event_loop()返回线程绑定的事件循环,它不返回当前运行的事件循环。

The combination of these behaviors is super confusing for a few reasons. First of all you need to know that these functions are delegates to the underlying event loop policy which is globally set. The default is to bind the event loop to the thread. Alternatively one could in theory bind the event loop to a greenlet or something similar if one would so desire. However it’s important to know that library code does not control the policy and as such cannot reason that asyncio will scope to a thread.

由于某些原因,这些行为的组合非常令人困惑。 首先,您需要知道这些功能是对全局设置的基础事件循环策略的委托。 默认设置是将事件循环绑定到线程。 或者,理论上可以将事件循环绑定到greenlet或类似的东西(如果有人愿意)。 但是,重要的是要知道库代码不控制策略,因此不能推理asyncio将限制在线程范围内。

Secondly asyncio does not require event loops to be bound to the context through the policy. An event loop can work just fine in isolation. However this is the first problem for library code as a coroutine or something similar does not know which event loop is responsible for scheduling it. This means that if you call asyncio.get_event_loop() from within a coroutine you might not get the event loop back that ran you. This is also the reason why all APIs take an optional explicit loop parameter. So for instance to figure out which coroutine is currently running one cannot invoke something like this:

其次,异步不需要通过策略将事件循环绑定到上下文。 事件循环可以很好地隔离工作。 但是,这是库代码作为协程的第一个问题,或者类似原因不知道哪个事件循环负责调度它。 这意味着,如果您在协程中调用asyncio.get_event_loop() ,则可能无法获得运行您的事件循环。 这也是所有API都采用可选的显式循环参数的原因。 因此,例如要弄清当前正在运行哪个协程,就不能调用如下代码:

def def get_taskget_task ():
    ():
    loop loop = = asyncioasyncio .. get_event_loopget_event_loop ()
    ()
    trytry :
        :
        return return asyncioasyncio .. TaskTask .. get_currentget_current (( looploop )
    )
    except except RuntimeErrorRuntimeError :
        :
        return return None
None

Instead the loop has to be passed explicitly. This furthermore requires you to pass through the loop explicitly everywhere in library code or very strange things will happen. Not sure what the thinking for that design is but if this is not being fixed (that for instance get_event_loop() returns the actually running loop) then the only other change that makes sense is to explicitly disallow explicit loop passing and require it to be bound to the current context (thread etc.).

相反,必须显式地传递循环。 此外,这还要求您在库代码中的任何地方显式地通过循环,否则将发生非常奇怪的事情。 不确定该设计的思路是什么,但是如果此设计不固定(例如get_event_loop()返回实际运行的循环),那么唯一有意义的其他更改是显式禁止显式循环传递并要求将其绑定到当前上下文(线程等)。

Since the event loop policy does not provide an identifier for the current context it also is impossible for a library to “key” to the current context in any way. There are also no callbacks that would permit to hook the tearing down of such a context which further limits what can be done realistically.

由于事件循环策略未提供当前上下文的标识符,因此库也不可能以任何方式“键入”当前上下文。 也没有回调允许钩住此类上下文的删除,这进一步限制了可以实际执行的操作。

待办事项和协程 (Awaitables and Coroutines)

In my humble opinion the biggest design mistake of Python was to overload iterators so much. They are now being used not just for iteration but also for various types of coroutines. One of the biggest design mistakes of iterators in Python is that StopIteration bubbles if not caught. This can cause very frustrating problems where an exception somewhere can cause a generator or coroutine elsewhere to abort. This is a long running issue that Jinja for instance has to fight with. The template engine internally renders into a generator and when a template for some reason raises a StopIteration the rendering just ends there.

在我的拙见中,Python的最大设计错误是使迭代器如此重载。 它们现在不仅用于迭代,而且还用于各种类型的协程。 Python中迭代器的最大设计错误之一是StopIteration如果不被捕获就会冒泡。 这会导致非常令人沮丧的问题,其中某个地方的异常可能导致其他地方的生成器或协程异常终止。 例如,这是一个长期运行的问题,Jinja必须解决。 模板引擎在内部渲染到生成器中,并且当模板由于某种原因引发StopIteration时 ,渲染就在那里结束。

Python is slowly learning the lesson of overloading this system more. First of all in 3.something the asyncio module landed and did not have language support. So it was decorators and generators all the way down. To implemented the yield from support and more, the StopIteration was overloaded once more. This lead to surprising behavior like this:

Python正在慢慢学习使系统更多重载的教训。 首先,在3.some异步模块降落并且没有语言支持。 因此一直都是装饰器和生成器。 为了实现支持和其他方面的收益StopIteration再次过载。 这将导致令人惊讶的行为,如下所示:

No error, no warning. Just not the behavior you expect. This is because a return with a value from a function that is a generator actually raises a StopIteration with a single arg that is not picked up by the iterator protocol but just handled in the coroutine code.

没有错误,没有警告。 只是不是您期望的行为。 这是因为从生成器的函数返回值实际上会引发一个带有单个arg的StopIteration ,该arg不是由迭代器协议选择的,而是仅在协程代码中处理的。

With 3.5 and 3.6 a lot changed because now in addition to generators we have coroutine objects. Instead of making a coroutine by wrapping a generator there is no a separate object which creates a coroutine directly. It’s implemented by prefixing a function with async. For instance async def x() will make such a coroutine. Now in 3.6 there will be separate async generators that will raise AsyncStopIteration to keep it apart. Additionally with Python 3.5 and later there is now a future import (generator_stop) that will raise a RuntimeError if code raises StopIteration in an iteration step.

3.5和3.6进行了很多更改,因为现在除了生成器之外,我们还有协程对象。 不需要通过包装生成器来制作协程,没有一个单独的对象可以直接创建协程。 通过在函数前面加上async来实现 。 例如async def x()会生成这样的协程。 在3.6版本中,现在将有单独的异步生成器,这些生成器将引发AsyncStopIteration来使其分开。 此外,在Python 3.5和更高版本中,现在有一个将来的导入( generator_stop ),如果代码在迭代步骤中提高StopIteration的话,它将引发RuntimeError

Why am I mentioning all this? Because the old stuff does not really go away. Generators still have send and throw and coroutines still largely behave like generators. That is a lot of stuff you need to know now for quite some time going forward.

我为什么要提所有这些? 因为旧的东西并没有真正消失。 生成器仍然具有发送抛出功能,并且协程在很大程度上仍像生成器一样工作。 在未来相当长的时间内,您现在需要了解很多东西。

To unify a lot of this duplication we have a few more concepts in Python now:

为了统一很多重复,我们现在在Python中有更多概念:

  • awaitable: an object with an __await__ method. This is for instance implemented by native coroutines and old style coroutines and some others.
  • coroutinefunction: a function that returns a native coroutine. Not to be confused with a function returning a coroutine.
  • a coroutine: a native coroutine. Note that old asyncio coroutines are not considered coroutines by the current documentation as far as I can tell. At the very least inspect.iscoroutine does not consider that a coroutine. It’s however picked up by the future/awaitable branches.
  • awaitable:具有__await__方法的对象。 例如,这是由本地协程和旧式协程等实现的。
  • coroutinefunction:返回本地协程的函数。 不要与返回协程的函数混淆。
  • 协程:原生协程。 请注意,据我所知,旧的异步协程在当前文档中不被视为协程。 至少inspect.iscoroutine不认为协程。 但是,它将由将来的/等待的分支机构来处理。

In particularly confusing is that asyncio.iscoroutinefunction and inspect.iscoroutinefunction are doing different things. Same with inspect.iscoroutine and inspect.iscoroutinefunction. Note that even though inspect does not know anything about asycnio legacy coroutine functions in the type check, it is apparently aware of them when you check for awaitable status even though it does not conform to __await__.

尤其令人困惑的是asyncio.iscoroutinefunctioninspect.iscoroutinefunction在做不同的事情。 与inspect.iscoroutineinspect.iscoroutine函数相同。 请注意,即使inspect在类型检查中对asycnio传统协程函数一无所知,但是当您检查等待状态时,即使它不符合__await__ ,它显然也会意识到它们。

协程包装 (Coroutine Wrappers)

Whenever you run async def Python invokes a thread local coroutine wrapper. It’s set with sys.set_coroutine_wrapper and it’s a function that can wrap this. Looks a bit like this:

每当您运行异步def时, Python都会调用线程本地协程包装器。 它是用sys.set_coroutine_wrapper设置的,并且可以包装此函数。 看起来像这样:

>>> import sys
>>> sys.set_coroutine_wrapper(lambda x: 42)
>>> async def foo():
...  pass
...
>>> foo()
__main__:1: RuntimeWarning: coroutine 'foo' was never awaited
42

In this case I never actually invoke the original function and just give you a hint of what this can do. As far as I can tell this is always thread local so if you swap out the event loop policy you need to figure out separately how to make this coroutine wrapper sync up with the same context if that’s something you want to do. New threads spawned will not inherit that flag from the parent thread.

在这种情况下,我永远不会真正调用原始函数,而只是给您提示它可以做什么。 据我所知,这始终是线程局部的,因此,如果您要执行此操作,则如果换出事件循环策略,则需要单独弄清楚如何使此协程包装与同一个上下文同步。 产生的新线程不会从父线程继承该标志。

This is not to be confused with the asyncio coroutine wrapping code.

这不要与asyncio协程包装代码混淆。

待定和期货 (Awaitables and Futures)

Some things are awaitables. As far as I can see the following things are considered awaitable:

有些事情是可以期待的。 据我所知,以下事情被认为是可以等待的:

  • native coroutines
  • generators that have the fake CO_ITERABLE_COROUTINE flag set (we will cover that)
  • objects with an __await__ method
  • 天然协程
  • 设置了伪造CO_ITERABLE_COROUTINE标志的生成器(我们将对此进行介绍)
  • __await__方法的对象

Essentially these are all objects with an __await__ method except that the generators don’t for legacy reasons. Where does the CO_ITERABLE_COROUTINE flag come from? It comes from a coroutine wrapper (now to be confused with sys.set_coroutine_wrapper) that is @asyncio.coroutine. That through some indirection will wrap the generator with types.coroutine (to to be confused with types.CoroutineType or asyncio.coroutine) which will re-create the internal code object with the additional flag CO_ITERABLE_COROUTINE.

从本质上讲,这些都是带有__await__方法的对象,除了生成器不是出于遗留原因之外。 CO_ITERABLE_COROUTINE标志来自何处? 它来自于@ asyncio.coroutine的协程包装器(现在与sys.set_coroutine_wrapper混淆)。 通过某种间接方式,将使用类型.coroutine (与类型.CoroutineTypeasyncio.coroutine混淆)包装生成器,这将使用附加标志CO_ITERABLE_COROUTINE重新创建内部代码对象。

So now that we know what those things are, what are futures? First we need to clear up one thing: there are actually two (completely incompatible) types of futures in Python 3. asyncio.futures.Future and concurrent.futures.Future. One came before the other but they are also also both still used even within asyncio. For instance asyncio.run_coroutine_threadsafe() will dispatch a coroutine to a event loop running in another thread but it will then return a concurrent.futures.Future object instead of a asyncio.futures.Future object. This makes sense because only the concurrent.futures.Future object is thread safe.

那么,既然我们知道那些东西是什么,什么是期货? 首先我们要澄清一两件事:其实有两个(完全不兼容)类型在Python 3 asyncio.futures.Futureconcurrent.futures.Future期货。 一个先于另一个,但即使在asyncio中,它们也都仍在使用。 例如, asyncio.run_coroutine_threadsafe()会将协程分派到在另一个线程中运行的事件循环,但它将返回并发 .futures.Future对象而不是asyncio.futures.Future对象。 这是有道理的,因为只有current.futures.Future对象是线程安全的。

So now that we know there are two incompatible futures we should clarify what futures are in asyncio. Honestly I’m not entirely sure where the differences are but I’m going to call this “eventual” for the moment. It’s an object that eventually will hold a value and you can do some handling with that eventual result while it’s still computing. Some variations of this are called deferreds, others are called promises. What the exact difference is is above my head.

因此,既然我们知道有两个不兼容的期货,就应该澄清一下异步中的期货。 坦白地说,我不确定这些差异在哪里,但是现在我将其称为“最终”。 这是一个最终将拥有值的对象,您可以在仍在计算时对该最终结果进行一些处理。 此方法的某些变体称为“递延”,其他则称为“应许”。 确切的区别在我头上。

What can you do with a future? You can attach a callback that will be invoked once it’s ready or you can attach a callback that will be invoked if the future fails. Additionally you can await it (it implements __await__ and is thus awaitable). Additionally futures can be cancelled.

未来你能做什么? 您可以附加一个回调,一旦准备好将被调用,或者可以附加一个在将来失败时将被调用的回调。 另外,您可以等待它(它实现了__await__ ,因此可以等待 )。 此外,可以取消期货。

So how do you get such a future? By calling asyncio.ensure_future on an awaitable object. This will also make a good old generator into such a future. However if you read the docs you will read that asyncio.ensure_future actually returns a Task. So what’s a task?

那么,您如何获得这样的未来? 通过在一个等待的对象上调用asyncio.ensure_future 。 这也将使一个好的旧发电机进入这样的未来。 但是,如果您阅读文档,将会看到asyncio.ensure_future实际上返回Task 。 那是什么任务呢?

任务 (Tasks)

A task is a future that is wrapping a coroutine in particular. It works like a future but it also has some extra methods to extract the current stack of the contained coroutine. We already saw the tasks mentioned earlier because it’s the main way to figure out what an event loop is currently doing via Task.get_current.

任务是特别包装协程的未来。 它像将来一样工作,但是它还有一些额外的方法可以提取当前包含的协程的堆栈。 我们已经看到了前面提到的任务,因为这是通过Task.get_current判断事件循环当前正在做什么的主要方式。

There is also a difference in how cancellation works for tasks and futures but that’s beyond the scope of this. Cancellation is its own entire beast. If you are in a coroutine and you know you are currently running you can get your own task through Task.get_current as mentioned but this requires knowledge of what event loop you are dispatched on which might or might not be the thread bound one.

取消在任务和期货上的工作方式也存在差异,但这超出了此范围。 取消是它自己的全部野兽。 如果您在协程中并且知道自己当前正在运行,则可以通过Task.get_current获得自己的任务(如上所述),但这需要了解您将在哪个事件循环上调度哪个事件循环,该事件循环可能是线程绑定的,也可能不是。

It’s not possible for a coroutine to know which loop goes with it. Also the Task does not provide that information through a public API. However if you did manage to get hold of a task you can currently access task._loop to find back to the event loop.

协程不可能知道哪个循环。 此外, 任务不会通过公共API提供该信息。 但是,如果您确实设法控制了任务,则当前可以访问task._loop以返回事件循环。

提手 (Handles)

In addition to all of this there are handles. Handles are opaque objects of pending executions that cannot be awaited but they can be cancelled. In particular if you schedule the execution of a call with call_soon or call_soon_threadsafe (and some others) you get that handle you can then use to cancel the execution as a best effort attempt but you can’t wait for the call to actually take place.

除了所有这些,还有句柄。 句柄是等待执行的不透明对象,无法等待但可以将其取消。 特别是,如果您计划使用call_sooncall_soon_threadsafe (以及其他一些调用)执行一次调用, 则会获得该句柄,然后可以使用该句柄尽最大努力取消执行,但您不能等待该调用实际发生。

执行者 (Executors)

Since you can have multiple event loops but it’s not obvious what the use of more than one of those things per thread is the obvious assumption can be made that a common setup is to have N threads with an event loop each. So how do you inform another event loop about doing some work? You cannot schedule a callback into an event loop in another thread and get the result back. For that you need to use executors instead.

由于您可以有多个事件循环,但是每个线程不止一个使用多个事物是不明显的,因此可以明显地假设一个常见的设置是每个事件循环都有N个线程。 那么,如何通知另一个事件循环进行某些工作呢? 您不能将回调安排到另一个线程的事件循环中并返回结果。 为此,您需要使用执行程序。

Executors come from concurrent.futures for instance and they allow you to schedule work into threads that itself is not evented. For instance if you use run_in_executor on the event loop to schedule a function to be called in another thread. The result is then returned as an asyncio coroutine instead of a concurrent coroutine like run_coroutine_threadsafe would do. I did not yet have enough mental capacity to figure out why those APIs exist, how you are supposed to use and when which one. The documentation suggests that the executor stuff could be used to build multiprocess things.

执行人来自concurrent.futures比如,他们让你安排工作纳入线程本身并没有事件触发。 例如,如果您在事件循环上使用run_in_executor来安排要在另一个线程中调用的函数。 然后将结果作为异步协程返回,而不是像run_coroutine_threadsafe这样的并发协程返回 。 我还没有足够的思维能力来弄清楚为什么存在这些API,应该如何使用以及何时使用这些API。 该文档建议执行程序的东西可以用来构建多进程的东西。

传输和协议 (Transports and Protocols)

I always though those would be the confusing things but that’s basically a verbatim copy of the same concepts in Twisted. So read those docs if you want to understand them.

我总是尽管会让人感到困惑,但这基本上是Twisted中相同概念的逐字复制。 因此,如果您想了解这些文档,请阅读这些文档。

如何使用异步 (How to use asyncio)

Now that we know roughly understand asyncio I found a few patterns that people seem to use when they write asyncio code:

现在我们大致了解了异步,我发现了人们在编写异步代码时似乎使用的一些模式:

  • pass the event loop to all coroutines. That appears to be what a part of the community is doing. Giving a coroutine knowledge about what loop is going to schedule it makes it possible for the coroutine to learn about its task.
  • alternatively you require that the loop is bound to the thread. That also lets a coroutine learn about that. Ideally support both. Sadly the community is already torn of what to do.
  • If you want to use contextual data (think thread locals) you are a bit out of luck currently. The most popular workaround is apparently atlassian’s aiolocals which basically requires you to manually propagate contextual information into coroutines spawned since the interpreter does not provide support for this. This means that if you have a utility library spawning coroutines you will lose context.
  • Ignore that the old coroutine stuff in Python exists. Use 3.5 only with the new async def keyword and co. In particular you will need that anyways to somewhat enjoy the experience because with older versions you do not have async context managers which turn out to be very necessary for resource management.
  • Learn to restart the event loop for cleanup. This is something that took me longer to realize than I wish it did but the sanest way to deal with cleanup logic that is written in async code is to restart the event loop a few times until nothing pending is left. Since sadly there is no common pattern to deal with this you will end up with some ugly workaround at time. For instance aiohttp‘s web support also does this pattern so if you want to combine two cleanup logics you will probably have to reimplement the utility helper that it provides since that helper completely tears down the loop when it’s done. This is also not the first library I saw do this
  • Working with subprocesses is non obvious. You need to have an event loop running in the main thread which I suppose is listening in on signal events and then dispatches it to other event loops. This requires that the loop is notified via asyncio.get_child_watcher().attach_loop(...).
  • Writing code that supports both async and sync is somewhat of a lost cause. It also gets dangerous quickly when you start being clever and try to support with and async with on the same object for instance.
  • If you want to give a coroutine a better name to figure out why it was not being awaited, setting __name__ doesn’t help. You need to set __qualname__ instead which is what the error message printer uses.
  • Sometimes internal type conversations can screw you over. In particular the asyncio.wait() function will make sure all things passed are futures which means that if you pass coroutines instead you will have a hard time finding out if your coroutine finished or is pending since the input objects no longer match the output objects. In that case the only real sane thing to do is to ensure that everything is a future upfront.
  • 将事件循环传递给所有协程。 这似乎是社区的一部分正在做的事情。 给出协程有关将要调度什么循环的知识,这使协程有可能了解其任务。
  • 或者,您需要将循环绑定到线程。 这也使协程知道了这一点。 理想情况下两者都支持。 可悲的是,社区已经为要做的事情感到痛苦。
  • 如果要使用上下文数据(认为是线程本地变量),那么您现在有点运气不好。 最流行的解决方法显然是atlassian的aiolocals ,它基本上需要您将上下文信息手动传播到生成的协程中,因为解释器不对此提供支持。 这意味着,如果您拥有一个生成协程的实用程序库,则将失去上下文。
  • 忽略Python中的旧协程物品。 仅将3.5与新的async def关键字和co一起使用。 尤其是,无论如何您都需要一定的经验,因为对于较旧的版本,您没有异步上下文管理器,这对于资源管理来说是非常必要的。
  • 了解重新启动事件循环进行清理。 这比我希望的要花更长的时间来实现,但是处理用异步代码编写的清除逻辑的最明智的方法是重新启动事件循环几次,直到没有任何待处理的事件为止。 遗憾的是,由于没有通用的模式可以解决此问题,因此您有时会遇到一些丑陋的解决方法。 例如, aiohttp的网络支持也采用这种模式,因此,如果您要组合两种清除逻辑,则可能必须重新实现其提供的实用程序帮助程序,因为该工具完成后会完全破坏循环。 这也不是我看到的第一个这样做的图书馆
  • 使用子流程并不明显。 您需要在主线程中运行一个事件循环,我想它正在监听信号事件,然后将其分派到其他事件循环。 这要求通过asyncio.get_child_watcher()。attach_loop(...)通知循环。
  • 编写同时支持异步和同步的代码有些失落。 它还得到,当你开始变得聪明,并试图 例如在同一对象上异步支持快速危险。
  • 如果您想给一个协程一个更好的名字来找出为什么不等待它,设置__name__并没有帮助。 您需要设置__qualname__ ,这是错误消息打印机使用的内容。
  • 有时内部类型的对话可能会使您烦恼。 特别是asyncio.wait()函数将确保传递的所有东西都是期货,这意味着如果您传递协程,则将很难找出协程是否完成或挂起,因为输入对象不再与输出对象匹配。 在那种情况下,唯一真正理智的事情就是确保所有事情都是未来的前期准备。

上下文数据 (Context Data)

Aside from the insane complexity and lack of understanding on my part of how to best write APIs for it my biggest issue is the complete lack of consideration for context local data. This is something that the node community learned by now. continuation-local-storage exists but has been accepted as implemented too late. Continuation local storage and similar concepts are regularly used to enforce security policies in a concurrent environment and corruption of that information can cause severe security issues.

除了疯狂的复杂性和缺乏对如何最好地编写API的理解之外,我最大的问题是完全不考虑上下文本地数据。 这是节点社区到目前为止学到的东西。 存在continuation-local-storage ,但由于实施太晚而被接受。 连续本地存储和类似概念通常用于在并发环境中实施安全策略,并且该信息的损坏可能会导致严重的安全问题。

The fact that Python does not even have any store at all for this is more than disappointing. I was looking into this in particular because I’m investigating how to best support Sentry’s breadcrumbs for asyncio and I do not see a sane way to do it. There is no concept of context in asyncio, there is no way to figure out which event loop you are working with from generic code and without monkeypatching the world this information will not be available.

Python对此根本没有任何存储的事实令人失望。 我之所以特别关注这个问题,是因为我正在研究如何最好地支持Sentry的面包屑支持asyncio,但我没有看到一种明智的方法。 在asyncio中没有上下文的概念,没有办法从通用代码中确定您正在使用哪个事件循环,并且在不了解世界的情况下,此信息将不可用。

Node is currently going through the process of finding a long term solution for this problem. That this is not something to be left ignored can be seen by this being a recurring issue in all ecosystems. It comes up with JavaScript, Python and the .NET environment. The problem is named async context propagation and solutions go by many names. In Go the context package needs to be used and explicitly passed to all goroutines (not a perfect solution but at least one). .NET has the best solution in the form of local call contexts. It can be a thread context, an web request context, or something similar but it’s automatically propagating unless suppressed. This is the gold standard of what to aim for. Microsoft had this solved since more than 15 years now I believe.

Node目前正在为该问题寻找长期解决方案 。 在所有生态系统中,这是一个反复出现的问题,可以看出这不是一个不容忽视的事情。 它带有JavaScript,Python和.NET环境。 这个问题被称为异步上下文传播 ,解决方案有很多名称。 在Go中,需要使用上下文包并将其显式传递给所有goroutine(不是完美的解决方案,但至少是一个解决方案)。 .NET具有本地调用上下文形式的最佳解决方案。 它可以是线程上下文,Web请求上下文或类似内容,但是除非被抑制,否则它将自动传播。 这是目标的黄金标准。 我相信,微软已经解决了这一问题,原因已经超过15年了。

I don’t know if the ecosystem is still young enough that logical call contexts can be added but now might still be the time.

我不知道生态系统是否还很年轻,可以添加逻辑调用上下文,但现在可能仍然是时候。

个人想法 (Personal Thoughts)

Man that thing is complex and it keeps getting more complex. I do not have the mental capacity to casually work with asyncio. It requires constantly updating the knowledge with all language changes and it has tremendously complicated the language. It’s impressive that an ecosystem is evolving around it but I can’t help but get the impression that it will take quite a few more years for it to become a particularly enjoyable and stable development experience.

那个东西很复杂,而且还在变得越来越复杂。 我没有能力随意地使用asyncio。 它要求随着所有语言的变化不断地更新知识,并且使语言变得极为复杂。 令人印象深刻的是,一个生态系统正在围绕着它发展,但我不禁给人一种印象,要成为一个特别令人愉悦和稳定的开发经验,还需要几年的时间。

What landed in 3.5 (the actual new coroutine objects) is great. In particular with the changes that will come up there is a sensible base that I wish would have been in earlier versions. The entire mess with overloading generators to be coroutines was a mistake in my mind. With regards to what’s in asyncio I’m not sure of anything. It’s an incredibly complex thing and super messy internally. It’s hard to comprehend how it works in all details. When you can pass a generator, when it has to be a real coroutine, what futures are, what tasks are, how the loop works and that did not even come to the actual IO part.

降落在3.5(实际的新协程对象)中的东西很棒。 特别是随着即将发生的更改,我希望在较早的版本中会有一个明智的基础。 在我看来,用过载发生器作为协程的整个混乱是一个错误。 关于异步中的内容,我不确定。 这是一件令人难以置信的事情,内部杂乱无章。 很难理解其所有细节的工作方式。 当您可以通过一个生成器时,必须是一个真正的协程,什么是期货,什么任务,循环如何工作,甚至还没有真正的IO部分。

The worst part is that asyncio is not even particularly fast. David Beazley’s live demo hacked up asyncio replacement is twice as fast as it. There is an enormous amount of complexity that’s hard to understand and reason about and then it fails on it’s main promise. I’m not sure what to think about it but I know at least that I don’t understand asyncio enough to feel confident about giving people advice about how to structure code for it.

最糟糕的是,异步并不是特别快。 大卫·比兹利(David Beazley)的现场演示破解了异步替代品的运行速度是其两倍。 存在大量难以理解和推理的复杂性,然后它未能兑现其主要承诺。 我不确定该怎么考虑,但至少我知道我对异步的理解不足,无法自信地向人们提供有关如何为其构建代码的建议。

翻译自: https://www.pybloggers.com/2016/10/i-dont-understand-pythons-asyncio/

你可能感兴趣的:(编程语言,python,人工智能,java,面试)