How can I find all registered ENS domains and subdomains?

Question

I would like to determine all of the active ENS domains and their subdomains, from ".eth" on down.

I've read over the ENS spec and searched the web, but I haven't found a way to do it.

So I suspect that either it's not possible, or else really hard?

Also, so far I've been using go-ethereum and github.com/wealdtech/go-ens in trying to do this work.

Thanks for any ideas. Robert

Answer 1

Well I'm super late to this, but since no one ever responded I'll give it a shot! (FYI, the answer is "really hard")

The thing to realize about ENS is that domains aren't actually stored in plaintext but as hashes (the function they use is namehash). Read this: https://www.npmjs.com/package/eth-ens-namehash

So in order to do this, you would need to:

1. Look at the ENS smart contract to determine all the addresses that have a domain.
2. Retrieve the hash that these addresses have registered
3. Determine what possible human-readable name could have resulted in this hash

3 is the part that will surely mess you up, because you basically have no other way to do it other than pre-computing hashes of known values and looking for a hit among all known eth addresses. (brute force)

However, this is SOMEWHAT doable because of a couple of contraints:

1. The number of registered domains IS PUBLIC at all times (thanks blockchain!), so you will always know both the number of addresses and how many of them you have decoded into human-readable.
2. The number of registered domains is still relatively small (40kish), so brute force isn't completely out of the realm of possibility
3. It isn't practical to register an .eth longer than the address that registered it... so at least you can set an upper bound on your dictionary that should cover 99% of registration cases.

Answer 2

Assuming that every new domain is created with the setSubnodeRecord() function of the ENSRegistry contract (which I believe is the case) then one approach is to use the NewOwner(bytes32 indexed node, bytes32 indexed label, address owner) logs. By querying all the NewOwner logs which have a given node parameter, you can obtain all of the sub-nodes of that node. By repeatedly applying this approach, you can obtain the entire tree of nodes from some root node (e.g. .eth)

ENS contract code source: https://github.com/ensdomains/ens/blob/ff0f41747c05f1598973b0fe7ad0d9e09565dfcd/contracts/ENSRegistry.sol#L53

Answer 3

Using Dune Analytics, you can query, chart, list, and visualize on-chain data from contracts in a number of different ways. Their documentation is great, and allows you to pull essentially any data from any contract that has been verified.

This Query, ENS released/registered names contains all registered ENS domains, ones that have just been released, and other data about the core ENS contracts.

If you don't need all of the additional data or would like to format it differently, you can click on the query title for Registered Domains and see the code for the query which you can take and change to meet your needs. Although the last link there for just the registered domain query is a very complete one.

Answer 4

You can also write a webscraper with Python and copy a list of potential ENS domains into a variable and then use the urllib module to check each ENS name separately on Etherscan. Here's a simple script from Finxter:

from urllib.request import Request, urlopen
import time

# Copy your list of names here:
list_of_names = '''
Google
Apple
Amazon
Microsoft
Tencent
Facebook
Visa
McDonald's
Alibaba
AT&T
IBM
Verizon
Marlboro
Coca-Cola
Mastercard
UPS
SAP
Wells Fargo
Disney
The Home Depot
China Mobile
ICBC
Starbucks
Xfinity
Deutsche Telekom
Louis Vuitton
Spectrum
GE
Nike
PayPal
Walmart
Accenture
Samsung
Moutai
American Express
Toyota
Vodafone
Intel
Hermes
Budweiser
Baidu
Zara
Ping An
L'Oreal Paris
Oracle
Mercedes-Benz
BMW
Huawei
China Construction Bank
HSBC
YouTube
RBC
Movistar
Gucci
NTT
FedEx
Cisco
Citi
JD.com
HDFC Bank
Netflix
DHL
Shell
Pampers
Orange
TD
Chase
Commonwealth Bank of Australia
Agricultural Bank of China
Subway
Colgate
Costco
J.P. Morgan
ExxonMobil
Adobe
IKEA
Bank of America
Salesforce
China Life
US Bank
Uber
Siemens
LinkedIn
Bank of China
Gillette
AIA
KFC
Ebay
HP
SF Express
Instagram
ANZ
ALDI
BT
Lowe's
Ford
Honda
Pepsi
BCA
Adidas
'''

# Automatically extract domain names from list
names = [x.strip().lower().replace("'", '').replace(' ', '').replace('.','') + '.eth' for x in list_of_names.split()]
print(names)

# For checking the .eth domain name
url = 'https://etherscan.io/enslookup-search?search='
negative_text = b'The domain name entered is either not registered on ENS or not currently supported by Etherscan.'

# Keep track of all free domain names
free_domains = []

for name in names:
  ens_url = url + name
  req = Request(
    url=ens_url, 
    headers={'User-Agent': 'Mozilla/5.0'}
  )
  webpage = urlopen(req).read()
  if negative_text in webpage:
    free_domains.append(name)
    print(name, ' available')
  else:
    print('xxx ', name, ' xxx')
  time.sleep(1)

print('Free Domains: ', '\n'.join(free_domains))

For a detailed explanation, see the article.