m4us.core.interfaces

Provides the interface definitions for all important core objects.

Inheritance Diagram

Members

interface IContainer

Interface that defines a container.

Containers are responsible for containing ICoroutine objects and the links that connect them both to each other and to the container itself, if appropriate.

All containers should calculate and provide IContainer.coroutines and IContainer.links attributes before they are used. The contents of IContainer.coroutines should be able to be passed to the appropriate IScheduler.register() method and the contents of IContainer.links should be able to be passed to the appropriate IPostOffice.register() method.

Additionally, containers can contain other IContainer objects, in other words other containers. In such a case, IContainer.coroutines and IContainer.links should include the coroutines and links from the sub-containers, excluding any duplicates from the other coroutines and links in the parent container.

Note

Containers will usually provide ICoroutine as well, but that is not strictly required by this interface. The calculation and use of the above-mentioned two attributes makes it not a strict requirement.

coroutines

All the contained coroutines to be added to the scheduler.

The coroutines of any sub-containers should also be included for convenience.

The contained coroutines can be added to the scheduler with code similar to scheduler.add(*container.coroutines).

Type :collections.Sequence

See also

IScheduler.register() for details about adding coroutines to schedulers.

links

All the post office links to be added to the post office.

The links of any sub-containers should also be included for convenience.

The links between the contained coroutines can be added to the post office with code similar to post_office.register(*container.links).

Type :collections.Sequence

See also

IPostOffice.register() for details about adding links to post offices.

interface IContainerFactory

Interface for callables that return IContainer objects.

__call__(*args, **kwargs)

Return the desired container.

Parameters:

• args (collections.Sequence) -- Any positional arguments that the factory may accept.
• kwargs (collections.Mapping) -- Any keyword arguments that the factory may accept.

Returns:

The desired container.

Return type:

IContainer

Raises m4us.core.exceptions.InvalidLinkError:  

If any of the given or calculated links are invalid.

interface ICoroutine

Interface that defines a Python coroutine.

Python coroutines are defined in PEP 342.

Note

While Python coroutines are an extension of generators, and as such define a next() method, this interface does not include the next() method as it really only makes sense in the context of actual generators rather than for coroutines.

Note

By default, all Python coroutines (i.e. types.GeneratorType) have been automatically configured to provide ICoroutine. This makes working with Python coroutines more natural.

close()

Terminate the coroutine.

This method sends the GeneratorExit exception into the coroutine, which is expected to perform any necessary clean up and shutdown work and then terminate.

Note

After this method is called, all subsequent calls to send() must result in a StopIteration exception being raised.

send(message)

Send the message to the coroutine.

Parameters:message -- The message object to send into the coroutine. For this project it will normally be a 2-tuple of the inbox and the message. Returns:Any message yielded by the coroutine. For this project it will normally be a 2-tuple of the outbox and the message. Raises exceptions.StopIteration:  If the coroutine has terminated, either by returning (as opposed to yielding) or because the close() method was called.

Note

The first call to this method must pass None as the sole argument. This activates the coroutine. The response from the first call is undefined.

throw(exception)

Send the exception to the coroutine.

The coroutine is expected to raise the exception in it's context at the reception point. It may then catch it and handle it or not.

Parameters:exception (exceptions.Exception) -- The exception to send to the coroutine. Returns:If the exception is caught, then any message yielded by the coroutine. Raises exceptions.Exception:  Any uncaught exception passed in via this method.

interface ICoroutineFactory

Interface for callables that return ICoroutine objects.

__call__(*args, **kwargs)

Return the desired coroutine.

Parameters:

• args (collections.Sequence) -- Any arguments to pass to the factory callable.
• kwargs (collections.Mapping) -- Keywords arguments to pass to the factory callable.

Returns:

The desired coroutine.

Return type:

ICoroutine

interface IMessage

Interface for special messages passed between coroutines.

interface IMessageFactory

Interface for callables that return IMessage objects.

__call__(**kwargs)

Return the desired message.

The callable should only be called with keyword arguments.

Parameters:kwargs (collections.Mapping) -- Keywords arguments to pass to the factory callable. Returns:The desired message object. Return type:IMessage

Note

All unrecognized keyword arguments should be set as attributes on the IMessage object.

interface INotLazy

Marker interface signalling that a coroutine is not lazy.

A coroutine is called lazy, if it should only be executed when there are new incoming messages for it. It is not lazy if the coroutine should be executed even when there are no incoming messages (i.e. for polling, etc.)

INotLazy coroutines will receive None objects as messages on their control inbox when there are no other messages to for them.

Note

By default, all objects providing ICoroutine are presumptively lazy. They need to provide this interface to indicate otherwise. This is more efficient and more natural for coroutines. It also more accurately distinguishes them from regular generators.

interface IPostOffice

Interface that defines a post office.

Post offices are objects that are responsible for delivering posted messages from inboxes to outboxes. They are also responsible for keeping track of the links between mailboxes.

That said, message posting and retrieval (and subsequent delivery to coroutines) is someone else's responsibility. Usually it is the job of the scheduler.

retrieve(sink)

Retrieve all outstanding messages for a sink coroutine.

As messages are posted, they are accumulated in message queues based on the registered links. When this method is called, all the accumulated messages for the given sink coroutine are returned and the message queue for that sink is emptied.

If there are no outstanding messages waiting in the message queue, then an empty iterable is returned.

Parameters:sink (m4us.core.interfaces.ICoroutine) -- The sink coroutine for which to retrieve messages. Returns:All outstanding messages in the form of (inbox, message). Return type:collections.Iterable Raises m4us.core.exceptions.NotASinkError:  If the given coroutine is not registered as a sink in any of the registered links.

Note

It is the responsibility of the caller to make sure all the retrieved messages are delivered to the sink coroutine.

unregister(first_link, *other_links)

Unregister previously registered links.

This is the opposite of the unregister() method.

Parameters:

• first_link (tuple) -- The first link to unregister.
• other_links (collections.Sequence) -- Any other links to unregister.

Each link should be a 4-tuple in the same format as defined in the register() method documentation.

Raises m4us.core.exceptions.NoLinkError:  If a link was not previously registered, unless the post office was created with unlink_ignores_missing=True.

Note

Unregistering one link should not affect any other links involving the source outbox or the sink inbox.

Note

Any outstanding messages in the sink's inbox message queue should still be kept for delivery. The message queue should only be removed after all outstanding messages have been retrieved.

See also

The register() method for details on the format of a link.

See also

The IPostOfficeFactory interface for details on the unlink_ignores_missing parameter.

register(first_link, *other_links)

Register links between coroutines.

When a source coroutine's outbox is linked to a sink coroutine's inbox, any messages posted from the source's outbox will automatically be delivered to the message queue for the sink's inbox, where it can be later retrieved for delivery to the sink coroutine.

Parameters:

• first_link (tuple) -- The first link to register.
• other_links (collections.Sequence) -- Any other links to register.

Each link should be a 4-tuple in the form of (source, outbox, sink, inbox), where:

Parameters:

• source (m4us.core.interfaces.ICoroutine) -- The source coroutine to link.
• outbox (unicode) -- The source's outbox to link.
• sink (m4us.core.interfaces.ICoroutine) -- The sink coroutine to link.
• inbox (unicode) -- The sink's inbox to link.

Raises m4us.core.exceptions.LinkExistsError:  

If a source's outbox is already linked to a sink's inbox, unless the post office was created with link_ignores_duplicates=True.

See also

The IPostOfficeFactory interface for details on the link_ignores_duplicates parameter.

post(source, outbox, message)

Post a message from the source outbox.

When a message is posted, it is automatically sent to the message queues of all linked sink inboxes for later retrieval and delivery.

Parameters:

• source (m4us.core.interfaces.ICoroutine) -- The source coroutine that produced the message.
• outbox (unicode) -- The outbox on which the source sent the message.
• message -- The message that the source sent.

Raises m4us.core.exceptions.NoLinkError:  

If the source outbox has not already been registered as a source in a link.

interface IPostOfficeFactory

Interface for callables that return IPostOffice objects.

__call__(link_ignores_duplicates=False, unlink_ignores_missing=False)

Return the desired post office.

Parameters:

• link_ignores_duplicates (bool) -- If True, the IPostOffice.register() method becomes idempotent and will not raise an exception when adding an already added link.
• unlink_ignores_missing (bool) -- If True, the IPostOffice.unregister() method becomes idempotent and will not raise an exception when removing a link that was not already added.

Returns:

The desired post office.

Return type:

IPostOffice

interface IProducerFinished

Extends: m4us.core.interfaces.IShutdown

Interface for messages signalling that a producer is done.

interface IScheduler

Interface that defines a scheduler.

Scheduler objects are responsible for being the main program loop. Their job is to run coroutines. They are also responsible for retrieving and delivering messages from a post office to the coroutines, as well as posting back to the post office any messages emitted by the coroutines.

run(cycles=None)

Start the scheduler, running all registered coroutines.

This is the main execution loop.

Parameters:cycles (int or None) -- The number of cycles of the main loop to run through. One cycle is defined as a one call to the cycle() method. If None or not specified, then this method should run until all registered coroutines have shutdown (i.e. raised StopIteration).

Note

If cycles is specified, it is expected that subsequent calls to this method should continue from where the last call left off.

See also

The cycle() method for details on what a single cycle entails and what exceptions may be raised as a result.

unregister(first_coroutine, *other_coroutines)

Unregister one or more coroutines from the scheduler.

This is the opposite of the register() method.

Parameters:

• first_coroutine (m4us.core.interfaces.ICoroutine) -- The first coroutine to remove.
• other_coroutines (collections.Sequence) -- Any other coroutines to remove.

Raises m4us.core.exceptions.NotAddedError:  

If any given coroutine is not registered with the scheduler, unless the scheduler was created with remove_ignores_missing=True.

Note

This method should also call each coroutine's close() method.

See also

The ISchedulerFactory interface for details on the remove_ignores_missing parameter.

register(first_coroutine, *other_coroutines)

Register one or more coroutines with the scheduler.

In order for a scheduler to run a coroutine, it must first be added to (i.e. registered with) the scheduler.

Parameters:

• first_coroutine (m4us.core.interfaces.ICoroutine) -- The first coroutine to add.
• other_coroutines (collections.Sequence) -- Any other coroutines to add.

Raises:

• exceptions.TypeError -- If any given argument does not provide or cannot be adapted to ICoroutine.
• m4us.core.exceptions.DuplicateError -- If any given coroutine has already been added, unless the scheduler was created with add_ignores_duplicates=True.

Note

Schedulers are not required to preserve or guarantee any particular order in which the coroutines will be run. They may, however, choose to do so, if desired.

See also

The ISchedulerFactory interface for details on the add_ignores_duplicates parameter.

step()

Run one coroutine.

This is the smallest unit of execution. A single registered coroutine is run once for each message currently accumulated in its post office message queue. Any resulting messages are also posted back to the post office. This means that one call to this method can trigger repeated calls to the current coroutine, but only one coroutine should ever be executed.

If a coroutine is lazy, it should only be executed when there are messages waiting to be delivered to it (or it is in the "shutting down" state, see below). If it is not lazy (i.e. it provides the INotLazy marker interface) and there are no messages waiting (as is the case with producers, for example), then a None object should be sent as the message to the coroutine on its control inbox.

If a coroutine yields only a None object (without even an outbox), the None should just be discarded. This lets the coroutine indicate it has no message to send.

If a coroutine yields an IShutdown message, that message should be posted to the post office, like a normal message, to allow the the message to cascade to other coroutines. Additionally, if the coroutine yields an IShutdown message on it's signal outbox, it should then be considered in a "shutting down" state. Finally, if the yielded IShutdown cannot be posted to the post office (i.e. the post office raises an NoLinkError), the the message should just be discarded. This is so that consumer/sink coroutines can also emit IShutdown messages to signal that they are shutting down.

When in the "shutting down" state, no futher post office messages should be sent to the coroutine. Instead, an IShutdown message should be sent to it's control inbox every time this method is called, until the coroutine exits (i.e. raises StopIteration). This allows coroutines to emit additional messages before shutting down and ensures that lazy coroutines always get enough messages to be able to shutdown properly.

Finally, any coroutine that exits (i.e. raises StopIteration) should be removed from any run queues and have its close() method called. After that no messages should ever be sent to it again.

Raises:

• m4us.core.exceptions.NeverRunError -- If a coroutine is lazy but the post office says it is not configured to receive any messages (i.e. raises NotASinkError).
• m4us.core.exceptions.NoLinkError -- If the post office raises the exception and the message is not an IShutdown message. IShutdown messages are expected to be cascaded (even from consumers), so they do not cause the exception to be re-raised.

Note

At least one coroutine is expected to be added before calling this method. It is an error to do otherwise.

See also

INotLazy for details about the non-lazy coroutines.

See also

IPostOffice.post() for details on the NoLinkError exception.

cycle()

Cycle once through the main loop, running all eligible coroutines.

Schedulers represent the main execution loop. This method triggers a single cycle through that execution loop, calling the step() method as many times as is appropriate. Ideally all registered coroutines should have a chance to run.

See also

The step() method for details on how coroutines are run and on the exceptions that this method can raise as a result.

interface ISchedulerFactory

Interface for callables that return IScheduler objects.

__call__(post_office, add_ignores_duplicates=False, remove_ignores_missing=False)

Return the desired scheduler.

Parameters:

• post_office (m4us.core.interfaces.IPostOffice) -- The post office through which to post and retrieve messages.
• add_ignores_duplicates (bool) -- If True, the IScheduler.register() method becomes idempotent and will not raise an exception when adding an already added coroutine.
• remove_ignores_missing (bool) -- If True, the IScheduler.unregister() method becomes idempotent and will not raise an exception when removing a coroutine that was not already added.

Returns:

The desired scheduler.

Return type:

IScheduler

Raises exceptions.TypeError:  

If the given post office does not provide the IPostOffice interface.

interface IShutdown

Extends: m4us.core.interfaces.IMessage

Interface for messages that tell coroutines to shutdown.

Upon receipt of an IShutdown message, coroutines should clean up any loose ends, forward on the IShutdown message on it's signal outbox and then shutdown.