Using https://etherchain.org API

Question

When I request https://etherchain.org/api/difficulty using python 3.6 I get HTTPError: Forbidden. Does anyone know the solution?

import json
from urllib.request import urlopen
from pandas.io.json import json_normalize

r_difficulty_eth = 
urlopen('https://etherchain.org/api/difficulty').read().decode('UTF-8')

difficulty_eth_data = json.loads(r_difficulty_eth)

Answer 1

Etherchain is protected by CloudFlare. It tries to prevent automated requests to the system. Which seems silly for an API... Anyway, the way around it is to make Python act as though it's a regular web browser in the User-Agent request header.

import json
import urllib.request

request = urllib.request.Request(url='https://etherchain.org/api/difficulty', 
                             data=None,
                             headers={
                                'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/603.2.4 (KHTML, like Gecko) Version/10.1.1 Safari/603.2.4'
                            })

r_difficulty_eth = urllib.request.urlopen(request).read().decode('UTF-8')
difficulty_eth_data = json.loads(r_difficulty_eth)