Debugging tips
==============

.. warning::
    These notes exist to assist developers in debugging their hosts in real time.  Do not follow
    any advice here without being prepared for the consequences.  Requests for support will be void
    on improperly-tampered hosts.

Run with full output
--------------------

Launch Python in developer mode to get stdlib deprecations and asyncio resource debugging:

.. code-block:: shell

    $ python3 -X dev -X tracemalloc -m immp config.yml

Configure your logging for the runner to output at DEBUG level:

.. code-block:: yaml

    logging:
      version: 1
      filters:
        local:
          (): immp.LocalFilter
      handlers:
        console:
          class: logging.StreamHandler
          filters: [local]
        internal:
          class: logging.FileHandle
          filename: internal.log
      loggers:
        asyncio:
          level: DEBUG
      root:
        level: DEBUG
        handlers: [console, internal]
      disable_existing_loggers: false

This will print local debug output (from the ``immp`` namespace) to stdout, and everything (IMMP
hooks/plugs as well as their dependencies) to a log file called ``internal.log``.

Enable the async shell
----------------------

The :ref:`admins/hooks:Shell` is an invaluable tool for poking internals, so make sure to have it
enabled and accessible.

Checking plug tasks
-------------------

If a plug is no longer picking up messages or showing signs of life, it may have a background task
that failed without restarting.  For example, the :ref:`admins/plugs:Telegram` plug has a long-poll
task to receive updates, stored on the plug's :attr:`_receive` attribute:

.. code-block:: pycon

    >>> plug._receive
    <Task finished coro=<TelegramPlug._poll() done,
          defined at /opt/IMMP/immp/plug/telegram.py:1234>
          exception=RuntimeError('Catastrophic failure',)>

If the task is finished with an exception, call :meth:`asyncio.Task.result()` to get a traceback.
A restart of the plug (:meth:`.Plug.stop`, :meth:`.Plug.start`) should get things moving again.

Checking the stream
-------------------

If a plug is logging creation of its own messages, but the host stream isn't receiving them, check
if its messages are stuck in the queue, and refer to :attr:`.Host._stream`:

.. code-block:: pycon

    >>> plug._queue.qsize()
    1
    >>> host._stream
    <PlugStream: 0 done, 12 pending>

If all are pending, then every plug should be idle and awaiting a new message.  If more than one
are done for more than a moment, then one is likely wedged and holding up the others.  You can try
transplanting a new stream by re-calling :meth:`.Host.process`:

.. code-block:: pycon

    >>> host._stream = host._process = None
    >>> host._process = asyncio.ensure_future(host.process())

Replaying messages into a hook
------------------------------

First, fetch the message(s) you're missing:

.. code-block:: pycon

    >>> msgs = (await host['my-channel'].history())[-3:]

Then feed them into a hook:

.. code-block:: pycon

    >>> for msg in msgs:
    ...     await host['sync'].on_receive(msg, msg, True)

For a :class:`.SyncHook`, this will send any as-yet-unsynced messages to linked channels.