8 – edit the custom blocks of a node’s layout without having to first click the layout tab?

Using the core layout builder to give a node a custom layout, is pretty sweet. But if you want to edit a custom block that’s displayed on one of the nodes you have to first click “layout” and go to /node/123/layout and then click on the block you want to edit. I’m surprised there’s no contextual link on each block.

It’d be great to be able to click the block’s “edit me” link directly from the nodes view url instead of having to first go to the layout url. Is this possible to do with contrib modules or a different configuration of core itself?