LevelDB in Geth, key and values

Question

When parsing through the levelDB or RocksDB (Depending on the client you are using) there are string values representing the key and the value. These are both stored as Byte arrays as well.

My question is: When analyzing each entry, what does the key represent in the relation to the data stored? Is the key just a unique identifier for the data or is it part of the data itself? I am trying to decode the RLP of an entry and need to know what the significance of the key is to the data.

Answer 1

Let's start with having a look at what we get in geth console for block number 40.

Now let's consider a Go program. I Explained everything in code comments about how it is constructing keys and accessing values from Leveldb.

package main

import (
    "github.com/syndtr/goleveldb/leveldb"
    "fmt"
    "encoding/binary"
    "github.com/ethereum/go-ethereum/core/types"
    "bytes"
    "github.com/ethereum/go-ethereum/rlp"
)


var (
    headerPrefix        = []byte("h") // headerPrefix + num (uint64 big endian) + hash -> header
    numSuffix           = []byte("n") // headerPrefix + num (uint64 big endian) + numSuffix -> hash
)

func main() {

    // Connection to leveldb
    db, _ := leveldb.OpenFile("/home/kara/.ethereum/geth/chaindata", nil)

    // 40 to bytes (Big endian)
    blockNumber := make([]byte, 8)
    binary.BigEndian.PutUint64(blockNumber, uint64(40))

    fmt.Printf("Details of Blocknumber:- \nHex: %x \nBytes: %d\n\n\n", blockNumber, blockNumber)

    // create key to get hash (headerPrefix + num (uint64 big endian) + numSuffix)
    hashKey := append(headerPrefix, blockNumber...) // adding prefix
    hashKey = append(hashKey, numSuffix...)  // adding suffix

    fmt.Printf("Details of leveldb key for Block Hash:- \nType: %T  \nHex: %x \nbytes: %v \nLength:  %d\n\n\n", hashKey,hashKey,hashKey, len(hashKey))

    // Getting hash using hashKey
    blockHash, _ := db.Get(hashKey, nil)
    fmt.Printf("Details of Block hash:- \nType: %T \nHex: %x \nBytes: %v\n\n\n", blockHash, blockHash,blockHash)

    //Create key to get header (headerPrefix + num (uint64 big endian) + hash)
    headerKey := append(headerPrefix, blockNumber...) // adding prefix
    headerKey = append(headerKey, blockHash...) // adding suffix

    fmt.Printf("Details of leveldb key for Block Header:- \nType: %T  \nHex: %x \nVytes: %v \nLength:  %d\n\n\n", headerKey,headerKey,headerKey, len(headerKey))

    //get Block Header data from db
    blockHeaderData, _ := db.Get(headerKey, nil)

    fmt.Printf("Details of Raw Block Header:- \nType: %T  \nHex: %x \nBytes: %v \nLength:  %d\n\n\n", blockHeaderData,blockHeaderData,blockHeaderData, len(blockHeaderData))

    //new Blockheader type
    blockHeader := new(types.Header)
    fmt.Printf("Details of new Header Type:- \nType: %T  \nHex: %x \nValue: %v\n\n\n", blockHeader,blockHeader,blockHeader)

    // Read blockHeaderData in a tmp variable
    tmpByteData := bytes.NewReader(blockHeaderData)
    fmt.Printf("Details of tmpByteData:- \nType: %T  \nHex: %x \nValue: %v\n\n\n", tmpByteData,tmpByteData,tmpByteData)

    //Decode tmpByteData to new blockHeader
    rlp.Decode(tmpByteData, blockHeader)
    fmt.Printf("Details of Header for block number 40:- \nType: %T  \nHex: %x \nValue: %v\n\n\n", blockHeader,blockHeader,blockHeader)

}

Link to file So that you can see highlighted syntax. gethLevelDbKeys.go Now lets see the Output of above Program in console.

Details of Blocknumber:-
Hex: 0000000000000028
Bytes: [0 0 0 0 0 0 0 40]


Details of leveldb key for Block Hash:-
Type: []uint8
Hex: 6800000000000000286e
bytes: [104 0 0 0 0 0 0 0 40 110]
Length:  10


Details of Block hash:-
Type: []uint8
Hex: c1579a13cfd2bd9050a5615968e1e9ccc3ba751b3343756851ece8d4a852d084
Bytes: [193 87 154 19 207 210 189 144 80 165 97 89 104 225 233 204 195 186 117 27 51 67 117 104 81 236 232 212 168 82 208 132]


Details of leveldb key for Block Header:-
Type: []uint8
Hex: 680000000000000028c1579a13cfd2bd9050a5615968e1e9ccc3ba751b3343756851ece8d4a852d084
Vytes: [104 0 0 0 0 0 0 0 40 193 87 154 19 207 210 189 144 80 165 97 89 104 225 233 204 195 186 117 27 51 67 117 104 81 236 232 212 168 82 208 132]
Length:  41


Details of Raw Block Header:-
Type: []uint8
Hex: f90216a06417bd3e58fbb42842be357b1b44a91a44b4907be271bb87990fd6015c99c0f7a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794bb7b8287f3f0a933474a79eae42cbca977791171a058701736553780fd938e2195cbf4176cdf25935fae9c8e74d532bb3835aa37bea056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000085040d13f2e228821388808455ba43419e476574682f4c5649562f76312e302e302f6c696e75782f676f312e342e32a0c8e64c566145f082ec79dfc40759b3e4b23b607a4c05e4b96cd5bfeb1222e754881953d8d473454660
Bytes: [249 2 22 160 100 23 189 62 88 251 180 40 66 190 53 123 27 68 169 26 68 180 144 123 226 113 187 135 153 15 214 1 92 153 192 247 160 29 204 77 232 222 199 93 122 171 133 181 103 182 204 212 26 211 18 69 27 148 138 116 19 240 161 66 253 64 212 147 71 148 187 123 130 135 243 240 169 51 71 74 121 234 228 44 188 169 119 121 17 113 160 88 112 23 54 85 55 128 253 147 142 33 149 203 244 23 108 223 37 147 95 174 156 142 116 213 50 187 56 53 170 55 190 160 86 232 31 23 27 204 85 166 255 131 69 230 146 192 248 110 91 72 224 27 153 108 173 192 1 98 47 181 227 99 180 33 160 86 232 31 23 27 204 85 166 255 131 69 230 146 192 248 110 91 72 224 27 153 108 173 192 1 98 47 181 227 99 180 33 185 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 133 4 13 19 242 226 40 130 19 136 128 132 85 186 67 65 158 71 101 116 104 47 76 86 73 86 47 118 49 46 48 46 48 47 108 105 110 117 120 47 103 111 49 46 52 46 50 160 200 230 76 86 97 69 240 130 236 121 223 196 7 89 179 228 178 59 96 122 76 5 228 185 108 213 191 235 18 34 231 84 136 25 83 216 212 115 69 70 96]
Length:  537


Details of new Header Type:-
Type: *types.Header
Hex: 4865616465722863336264326430303734356330333034386135363136313436613936663566663738653534656662396535623034616632303863646166663666333833306565293a0a5b0a09506172656e74486173683a0920202020303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a09556e636c65486173683a0920202020303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a09436f696e626173653a0920202020303030303030303030303030303030303030303030303030303030303030303030303030303030300a09526f6f743a090920202020303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a095478536861090920202020303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a09526563656970745368613a0920202020303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a09426c6f6f6d3a09092020202030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a09446966666963756c74793a09202020203c6e696c3e0a094e756d6265723a0909202020203c6e696c3e0a094761734c696d69743a09202020203c6e696c3e0a09476173557365643a09202020203c6e696c3e0a0954696d653a0909202020203c6e696c3e0a0945787472613a0909202020200a094d69784469676573743a202020202020303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a094e6f6e63653a090920202020303030303030303030303030303030300a5d
Value: Header(c3bd2d00745c03048a5616146a96f5ff78e54efb9e5b04af208cdaff6f3830ee):
[
ParentHash:         0000000000000000000000000000000000000000000000000000000000000000
UncleHash:          0000000000000000000000000000000000000000000000000000000000000000
Coinbase:           0000000000000000000000000000000000000000
Root:               0000000000000000000000000000000000000000000000000000000000000000
TxSha               0000000000000000000000000000000000000000000000000000000000000000
ReceiptSha:         0000000000000000000000000000000000000000000000000000000000000000
Bloom:              00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Difficulty:         <nil>
Number:             <nil>
GasLimit:           <nil>
GasUsed:            <nil>
Time:               <nil>
Extra:
MixDigest:      0000000000000000000000000000000000000000000000000000000000000000
Nonce:              0000000000000000
]


Details of tmpByteData:-
Type: *bytes.Reader
Hex: &{f90216a06417bd3e58fbb42842be357b1b44a91a44b4907be271bb87990fd6015c99c0f7a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794bb7b8287f3f0a933474a79eae42cbca977791171a058701736553780fd938e2195cbf4176cdf25935fae9c8e74d532bb3835aa37bea056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000085040d13f2e228821388808455ba43419e476574682f4c5649562f76312e302e302f6c696e75782f676f312e342e32a0c8e64c566145f082ec79dfc40759b3e4b23b607a4c05e4b96cd5bfeb1222e754881953d8d473454660 0 -1}
Value: &{[249 2 22 160 100 23 189 62 88 251 180 40 66 190 53 123 27 68 169 26 68 180 144 123 226 113 187 135 153 15 214 1 92 153 192 247 160 29 204 77 232 222 199 93 122 171 133 181 103 182 204 212 26 211 18 69 27 148 138 116 19 240 161 66 253 64 212 147 71 148 187 123 130 135 243 240 169 51 71 74 121 234 228 44 188 169 119 121 17 113 160 88 112 23 54 85 55 128 253 147 142 33 149 203 244 23 108 223 37 147 95 174 156 142 116 213 50 187 56 53 170 55 190 160 86 232 31 23 27 204 85 166 255 131 69 230 146 192 248 110 91 72 224 27 153 108 173 192 1 98 47 181 227 99 180 33 160 86 232 31 23 27 204 85 166 255 131 69 230 146 192 248 110 91 72 224 27 153 108 173 192 1 98 47 181 227 99 180 33 185 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 133 4 13 19 242 226 40 130 19 136 128 132 85 186 67 65 158 71 101 116 104 47 76 86 73 86 47 118 49 46 48 46 48 47 108 105 110 117 120 47 103 111 49 46 52 46 50 160 200 230 76 86 97 69 240 130 236 121 223 196 7 89 179 228 178 59 96 122 76 5 228 185 108 213 191 235 18 34 231 84 136 25 83 216 212 115 69 70 96] 0 -1}


Details of Header for block number 40:-
Type: *types.Header
Hex: 4865616465722863313537396131336366643262643930353061353631353936386531653963636333626137353162333334333735363835316563653864346138353264303834293a0a5b0a09506172656e74486173683a0920202020363431376264336535386662623432383432626533353762316234346139316134346234393037626532373162623837393930666436303135633939633066370a09556e636c65486173683a0920202020316463633464653864656337356437616162383562353637623663636434316164333132343531623934386137343133663061313432666434306434393334370a09436f696e626173653a0920202020626237623832383766336630613933333437346137396561653432636263613937373739313137310a09526f6f743a090920202020353837303137333635353337383066643933386532313935636266343137366364663235393335666165396338653734643533326262333833356161333762650a095478536861090920202020353665383166313731626363353561366666383334356536393263306638366535623438653031623939366361646330303136323266623565333633623432310a09526563656970745368613a0920202020353665383166313731626363353561366666383334356536393263306638366535623438653031623939366361646330303136323266623565333633623432310a09426c6f6f6d3a09092020202030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030300a09446966666963756c74793a092020202031373339393238303335340a094e756d6265723a09092020202034300a094761734c696d69743a0920202020353030300a09476173557365643a0920202020300a0954696d653a090920202020313433383237303237330a0945787472613a090920202020476574682f4c5649562f76312e302e302f6c696e75782f676f312e342e320a094d69784469676573743a202020202020633865363463353636313435663038326563373964666334303735396233653462323362363037613463303565346239366364356266656231323232653735340a094e6f6e63653a090920202020313935336438643437333435343636300a5d
Value: Header(c1579a13cfd2bd9050a5615968e1e9ccc3ba751b3343756851ece8d4a852d084):
[
ParentHash:         6417bd3e58fbb42842be357b1b44a91a44b4907be271bb87990fd6015c99c0f7
UncleHash:          1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347
Coinbase:           bb7b8287f3f0a933474a79eae42cbca977791171
Root:               58701736553780fd938e2195cbf4176cdf25935fae9c8e74d532bb3835aa37be
TxSha               56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421
ReceiptSha:         56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421
Bloom:              00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Difficulty:         17399280354
Number:             40
GasLimit:           5000
GasUsed:            0
Time:               1438270273
Extra:              Geth/LVIV/v1.0.0/linux/go1.4.2
MixDigest:      c8e64c566145f082ec79dfc40759b3e4b23b607a4c05e4b96cd5bfeb1222e754
Nonce:              1953d8d473454660
]

This is just an example. If you want to know everything in detail. You can consider schema.go file in ethereum source code.

Answer 2

I think keys are needed to organize tree - to know how to move nodes in trie without knowledge of stored information, read about Patricia-Tree. In examples, they use keys to organize trie.