segregated witness – Parsing raw transaction vs. raw segwit transaction

I am writing a bitcoin blkXXXX.dat files parser for my own project. It works well with non-segwit transactions, but I have problems with SegWit transactions.
Currently I read: version (4 bytes), inputsConter (varint), read and parse all inputs, outputsCounter(varint), read and parse all outputs, locktime(4 bytes).
I do not need signatures, but I need to parse these transactions. Can you please help me fix my program?

Many thanks!

segregated witness – Why SegWit address fees are lower than Legacy ones?

Why are segwit transactions cheaper?

Segregated witness transactions are cheaper because they are designed to be.

In normal transactions, every byte has the same “cost”; they all count equally towards the 1000000 byte block limit. Segwit transactions on the contrary keep certain data (signatures etc) in a separate “witness” section, and that witness section is discounted: every witness byte only counts as 0.25 byte towards the block limit. Fees are determined by the market and not a system rule, but the fact that segwit transactions have some of their content discounted results in the market pricing them less as a result.

There is a reason why this discount is justified: in legacy transactions, creating a transaction output is significantly cheaper than spending one. This encourages unspendable dust: outputs that were created at a time when fees were low may become uneconomical to spend (= cost more to spend than they’re worth) when fees are high. This is a burden on the entire ecosystem, as full nodes (for now, at least) need to maintain fast access to the set of all unspent outputs.

Why are legacy transactions still in use?

Compatibility. There is little more to say about it: they’re still permitted because it would be impossible to get a change that broke people’s existing software without an extremely good reason. And people still use them, because they haven’t upgraded their infrastructure to use segwit transactions – the engineering costs to perform this upgrade may be (rightfully or not) considered higher than what would be gained by upgrading.

segregated witness – Which wallet libraries officially intend to use Bech32 addresses, and will the addresses generated by them be compatible?

Bech32 is an address format that was only recently proposed. While its design had input from multiple wallet authors, it is way too early to say anything about adoption.

It is important to realize that there is no hurry about this. Every usable address type is available through embedding in P2SH, which is compatible with every wallet created the past few years.

At some point we’ll need the higher efficiency and security that native SegWit outputs offer, but that is dependent on SegWit both activating, and being available in both sender and receiver wallets.

Disclaimer: I’m the author of the Bech32 proposal.

segregated witness – What is the prefix number, in bytes, of a native segwit address?

Native segwit addresses are not encoded using base58, but using a different encoding system called bech32. The details can be found in BIP 173. For the segwit v1+ addresses that will be used for Taproot, see the amended bech32m encoding in BIP 350.

In short, the addresses consist of:

  • A human readable part, which is bc for Bitcoin mainnet segwit addresses.
  • A separator (1)
  • A payload (storing the actual script data), consisting of:
    • A character for the witness version (q for witness v0, p for witness v1, …). See the BIPs for details on the character set.
    • A variable number of characters storing the witness program (32 characters for 20-byte witness programs like P2WPKH has; 52 characters for 32-byte witness programs like P2WSH or P2TR have). The 8-bit data is converted to 5-bit groups, and encoded using the bech32 character set.
  • 6 characters checksum, using the checksum algorithm specified in BIP173 or BIP350.

segregated witness – Is a P2WPKH address synonymous with a Bech32 address?

P2WPKH and bech32 are not quite synonymous.

Native segwit describes a type of output schema in which the scriptPubKey (the “locking script”) is composed of a witness version and a witness program. P2WPKH is a native segwit v0 output type.

Bech32 refers to an encoding scheme and an address format to represent a recipient’s scriptPubKey in out-of-band communication.

We use “bech32 addresses” to instruct spenders to create “P2WPKH outputs”:

  • Pay to Witness Public Key Hash (P2WPKH) and Pay to Witness Script Hash (P2WSH) are native segwit v0 outputs that get represented using bech32 addresses.
  • Pay to Public Key Hash (P2PKH) and Pay to Script Hash (P2SH) use base58 addresses.
  • Pay to Taproot (P2TR) and future native segwit output types will use bech32m addresses.

segregated witness – How can I generate SegWit addresses from a mnemonic using python?

So, I have written a script that generates BIP39 mnemonics. I now want to generate N native SegWit public keys from this mnemonic.

I have found this, but it generates legacy addresses. It uses the module bip32utils.

def bip39(mnemonic_words):
    mobj = mnemonic.Mnemonic("english")
    seed = mobj.to_seed(mnemonic_words)

    bip32_root_key_obj = bip32utils.BIP32Key.fromEntropy(seed)
    bip32_child_key_obj = bip32_root_key_obj.ChildKey(
        44 + bip32utils.BIP32_HARDEN
    ).ChildKey(
        0 + bip32utils.BIP32_HARDEN
    ).ChildKey(
        0 + bip32utils.BIP32_HARDEN
    ).ChildKey(0).ChildKey(0)

    return {
        'mnemonic_words': mnemonic_words,
        'addr': bip32_child_key_obj.Address(),
        'publickey': binascii.hexlify(bip32_child_key_obj.PublicKey()).decode(),
        'privatekey': bip32_child_key_obj.WalletImportFormat(),
        'coin': 'BTC'
    }

if __name__ == '__main__':
    seed = 'input display smile visa surround learn solar hero vacuum parrot cigar devote'
    pprint.pprint(bip39(seed))

I assume I need bip44utils or something, but it doesn’t exists. I have tried looking at bip-utils but I can’t work out how to convert my mnemonic to an address.

Does anyone know how I can convert a mnemonic to a SegWit address (in a way that I can change the derivation path to get multiple addresses)?

EDIT (SOLVED):

I found bitcoinlib can do this.

from bitcoinlib.wallets import Wallet

passphrase = 'input display smile visa surround learn solar hero vacuum parrot cigar devote'

w = Wallet.create("Wallet1", witness_type='segwit', keys=passphrase, network='bitcoin')
WalletKeys = (w.get_keys(number_of_keys=10))

for k in WalletKeys:
    print(k.address)

And now it works just as I wanted it to.

segregated witness – The example signature in BIP143 doesn’t validate?

In BIP143 the first example
has a witness signature and it doesn’t appear to validate/be correct?! I wrote a super simple python program to demonstrate

import ecdsa
import asn1

#Importing the 3 pieces of data from the example to byte arrays
pub = bytearray.fromhex("025476c2e83188368da1ff3e292e7acafcdb3566bb0ad253f62fc70f07aeee6357")
sighash = bytearray.fromhex("c37af31116d1b27caf68aae9e3ac82f1477929014d5b917657d0eb49478cb670")
dersigscript = bytearray.fromhex("304402203609e17b84f6a7d30c80bfa610b5b4542f32a8a0d5447a12fb1366d7f01cc44a0220573a954c4518331561406f90300e8f3358f51928d43c212a8caed02de67eebee")

#deocding the DEC encoding
decoder = asn1.Decoder()
decoder.start(bytes(dersigscript))
tag, sigscript = decoder.read()

#stripping off the script  potion so we just have a signature
sig = bytearray(sigscript)(2:66)

vk = ecdsa.VerifyingKey.from_string(pub,  curve=ecdsa.SECP256k1)
vk.verify(sig, sighash)

If anyone can shed some light on what stupid thing I am doing, or an example reference that breaks it down, so it works, that would be really helpful. I have also tried reversing byte order on all 8 combinations of the 3 inputs. That being said I have used the private and public key posted to do my own signatures/validation, so I am fairly confident they are in correct order, just less sure about the sig.

Thank you!!

segregated witness – bip143 example signature doesn’t validate?

in bip 143 the first example https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki
has a witness signature and it doesn’t appear to validate/be correct?! I wrote a super simple python program to demonstrate

import ecdsa
import asn1

#Importing the 3 pieces of data from the example to byte arrays
pub = bytearray.fromhex("025476c2e83188368da1ff3e292e7acafcdb3566bb0ad253f62fc70f07aeee6357")
sighash = bytearray.fromhex("c37af31116d1b27caf68aae9e3ac82f1477929014d5b917657d0eb49478cb670")
dersigscript = bytearray.fromhex("304402203609e17b84f6a7d30c80bfa610b5b4542f32a8a0d5447a12fb1366d7f01cc44a0220573a954c4518331561406f90300e8f3358f51928d43c212a8caed02de67eebee")

#deocding the DEC encoding
decoder = asn1.Decoder()
decoder.start(bytes(dersigscript))
tag, sigscript = decoder.read()

#stripping off the script  potion so we just have a signature
sig = bytearray(sigscript)(2:66)

vk = ecdsa.VerifyingKey.from_string(pub,  curve=ecdsa.SECP256k1)
vk.verify(sig, sighash)

If anyone can shed some light on what stupid thing I am doing, or an example reference that breaks it down so it works it would be really helpful. I should say I have also tried reversing byte order on all 8 combinations of the 3 inputs. That being said I have used the private and public key posted to do my own signatures/validation, so I am fairly confident they are in correct order, just lest sure about the sig.

Thank you!!

segregated witness – Are high-s ECDSA signatures forbidden in segwit witnesses?

I must have looked that up five times by now, but did segwit actually forbid high-s ECDSA signatures in witnesses, esp. in standard single-sig constructions such as P2SH-P2WPKH or P2WPKH? Or are high-s signatures still only non-standard even in segwit inputs?

(I’m aware that the transaction malleability problem due to low-s/high-s is mitigated by moving the signature into the witness, that’s not my question.)

segregated witness – generating a zprv from a base58 rootKey with bitcoinjs-lib

var bitcoin = require('bitcoinjs-lib');
let bip32 = require('bip32');

let rootKey = 'xprv9s21ZrQH143K32CfqoCQHtbKiEb5BqFazXX6jCtNCnuty3gUjxS4CsXWi9rcNyHdjDVPiC6P1bnyEZr2ioouRq56h6HAdwejeTty1BsSEtL'
const node = bip32.fromBase58(rootKey, bitcoin.networks.bitcoin);

//legacy account extended private key
console.log("LEGACY: " + node.derivePath("m/44'/0'/0").toBase58())
//xprv9y7nL2k2diLdA9bmop94K8Pec2WdL998PLdvTraxj5jiQ36gnLmhVJG4bY8FzgqZPaBM5HDBi9sgRK1ZCi14n5j4Pk1o6d3j4Y6T9NS4nA6
(works as expected)


//segwit account extended private key
//trying to get a segwit extended private key (zprv) using the same inputs as above.

I want to get a segwit/bech32 extended private key using the same root key as above. How can I do this using bitcoinjs-lib? BIP84 only works for mnemonics IIRC.