Rusty Russell discussed lnprototest (repo link) during the August 2020 Sydney Socratic Seminar.
What is lnprototest exactly?
It is a pure Python3 test system and some packages to interface with
the spec that currently live in the c-lightning repository. You run
lnprototest and it has these scripts that go “I will send this” and
you will send back this. It can keep state and does some quite
sophisticated things. It has a whole heap of scaffolding to understand
commitment transactions, anchor outputs and a whole heap of other
things. Then you write these scripts that say “If I send this it
should send this” or “If I send this instead…”. You create this DAG, a
graph of possible things that could happen and it runs through all of
them and checks that happens.
Can it be used to test an individual Lightning implementation or just interoperability between implementations?
It is really good for protocol development too not just testing
existing stuff. When you want to modify the spec you can write that
half and run it against your own node.
Since writing this answer a PR in the c-lightning repo from Lisa Neigut was merged to run lnprototest with every c-lightning CI build.
What does an implementation need to do to use lnprototest?
They need their own runner. Take the c-lightning one, it fires up a
bitcoind in regtest mode and fires up a c-lightning node. It is kind
of dumb. You have to have c-lightning run in developer mode because it
uses some weird hacks. We are slowly pulling those out. Ideally you’d
be able to run it in any off the shelf implementation. You have to get
your hands dirty a bit and write some Python for your implementation,
that is true.