Get Your Duolingo Stats

Unfortunately the interface to https://www.duolingo.com has changed so the examples in the blog are no longer working. I left the page up for reference and I’ll try and get more information on an updated Duolingo API.

Doulingo is one of the most popular applications for learning a new language. Duolingo has a good selection of languages and it’s free.

If you wanted to create your apps to look at your progress or do some comparisons with other Doulingo users it’s a little challenging because there isn’t a documented API (Application Programming Interface).

In this blog I wanted to look at two programming choices that are available. The first is a Web page (REST API) query and the second is a Python library.

It is important to note that last year the Doulingo interface changed so a lot of the documentation on the Internet is not 100% valid.

The Web Page (REST API) Query

If you enter : https://www.duolingo.com/users/a_user_name

You will see a very large amount of JSON (JavaScript Object Notation) text. This undocumented text is quite ugly.

web_restapi

It is possible to use some Python code to call the Web page, and try to make the JSON a little more readable. The code below uses the requests library to get the Web data, and the json library to format the web output (JSON) data.

import requests
import json

r = requests.get('https://www.duolingo.com/users/some_users')

jtext = json.loads(r.text)

with open('user_data.txt', 'w', encoding='utf8') as outfile:
     json.dump(jtext, outfile, sort_keys = False, indent = 4,
               ensure_ascii = False)

This example created an outfile that is much easier (but far from simple) to analyze. Below is a section of the output file where we can see the languages section broken out.

    "tts_base_url_http": "http://d7mj4aqfscim2.cloudfront.net/",
    "id": 141898265,
    "dict_base_url": "http://d2.duolingo.com/",
    "cohort": null,
    "daily_goal": 10,
    "delete-permissions": false,
    "ads_enabled": true,
    "languages": [         {
            "streak": 4,
            "language_string": "Greek",
            "points": 0,
            "learning": false,
            "language": "el",
            "level": 1,
            "current_learning": false,
            "sentences_translated": 0,
            "to_next_level": 60
        },
        {
            "streak": 4,
            "language_string": "Esperanto",
            "points": 0,
            "learning": false,
            "language": "eo",
            "level": 1,
            "current_learning": false,
            "sentences_translated": 0,
            "to_next_level": 60
        },

Show the Languages and Work History

By looking at the outfile the languages studied and the work history can be determined:

import requests
import json
from datetime import datetime

r = requests.get('https://www.duolingo.com/users/pete_xxxxx')

j = json.loads(r.text)

print ("Languages Studied\n")
for thelang in j["languages"]:
    if thelang["points"] > 0 :
        print (thelang["language_string"], ",Level = ", thelang["level"], ",Points=", thelang["points"])

print ("\n\nWork History\n")

for ical in j["calendar"]:
    ddate =  datetime.fromtimestamp(ical["datetime"] / 1e3)
    sdate = ddate.strftime("%m/%d/%Y")
    print(sdate, ", Improvement = ", ical['improvement'])

For my example this gave the following output:


Languages Studied

Indonesian ,Level = 6 ,Points= 570
Spanish ,Level = 13 ,Points= 5102
Chinese ,Level = 2 ,Points= 70
French ,Level = 9 ,Points= 1710
German ,Level = 6 ,Points= 616
Japanese ,Level = 10 ,Points= 2297

Work History

02/22/2019 , Improvement = 10
02/22/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/24/2019 , Improvement = 10

Duolingo Python Library

There is an unofficial Duolingo Python Library, the author did an excellent job of putting together some useful calls. Unfortunately since the Duolingo interface changed last year not all the functions are now available or working. However the key ones are still usable.

To install the library go to a terminal window or command prompt and enter:

pip install duolingo-api

A Python example would be:

import duolingo
from datetime import datetime

lingo  = duolingo.Duolingo('pete_xxxxx')

# get_user_info : works but outputs too much info
#print (lingo.get_user_info())

print ("Languages I am studying:")
print (lingo.get_languages(abbreviations=False))

print("\n\nLanguage Progress\n")
for ilang in lingo.get_languages(abbreviations=False):
       jlang = lingo.get_language_details(ilang)
       print( jlang["language_string"]," ,Points: ",jlang["points"], "Level = ", jlang["level"])

print("\n\nWork History\n")
for ical in lingo.get_calendar():
    ddate =  datetime.fromtimestamp(ical["datetime"] / 1e3)
    sdate = ddate.strftime("%m/%d/%Y")
    print(sdate, ", Improvement = ", ical['improvement'])

For my example the output is:


Languages I am studying:
['Indonesian', 'Spanish', 'Chinese', 'French', 'German', 'Japanese']

Language Progress

Indonesian ,Points: 570 Level = 6
Spanish ,Points: 5102 Level = 13
Chinese ,Points: 70 Level = 2
French ,Points: 1710 Level = 9
German ,Points: 616 Level = 6
Japanese ,Points: 2297 Level = 10

Work History

02/22/2019 , Improvement = 10
02/22/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/24/2019 , Improvement = 10

Summary

Now that we have a basic interface to Duolingo it’s possible to start doing some trends and charts. Below a is some simple chart code for language levels.

import duolingo
import numpy as np
import matplotlib.pyplot as plt

lingo  = duolingo.Duolingo('pete_xxxx')

thelang = lingo.get_languages(abbreviations=False)

thelevel = []
for ilang in lingo.get_languages(abbreviations=False):
       jlang = lingo.get_language_details(ilang)
       thelevel.append(jlang["level"])


plt.bar(thelang,thelevel, color='r')

plt.legend()
plt.xlabel('Language')
plt.ylabel('Level')

plt.title('My Duolingo Progress')

plt.show()

Duo_progress

Python talking to littleBits Cloud API

LittleBits is a set of components that allow kids to build their own electrical circuits. The Cloudbit will send and receive values from the Internet.

For this project we wanted to build some Python programs that could interact with the littleBits Cloud API.

littleBits Hardware Setup

cloudbit_setup

The Cloudbit enables an input to be sent to the Internet, and it will receive a value (output) from the Internet. For a test setup we used a dimmer bit to adjust the input value with a number bit to show the value. On the output side we used a bargraph bit. Other combinations of bits are possible. The key is to be able to test and see your input and output values.

Cloudbits Setup

For the Cloudbit setup you will need to create a sign-in on the littleBits web site. For details on how to setup your Cloudbit use the following link.

Python Basic Setup

The Cloud API is a RESTful interface, that uses http requests with some header definitions. Before you get started you’ll need your specific littleBits device ID and authorization token. In your specific Cloudbit definition go to the Settings icon, and get your Device ID and AccessToken.

littlebit_id

 

The Cloud API presently does not support a single point read (this might change), instead the input data point value is streamed about once a second.

The basic syntax in Python to get values from the  Cloud API will be:

import requests

requests.packages.urllib3.disable_warnings()

# Change with your Authorization code and deviceID
authToken = "4f3830b44e15fa0d"
deviceId = "00e04c"

headers = {"Authorization": "Bearer " + authToken,"Content-type": "application/json"}

# For a read 
littleBitsUrl = "https://api-http.littlebitscloud.cc/devices/" + deviceId + "/input"
r = requests.get(littleBitsUrl, headers=headers, stream=True)

#  For a write, note: a data body needs to be defined
# littleBitsUrl = "https://api-http.littlebitscloud.cc/devices/" + deviceId + "/output"
# body = {"percent": thevalue , "duration_ms": thetime} 
# thebody = json.dumps(body)
# r = requests.post(littleBitsUrl, data=thebody, headers=headers)

To do a read of the Cloud API an get request is done, to do a write a post request is used.

Python Read Value Example

The cloud API will stream the input value about every second in a format something like:

data: {"type":"input","timestamp":1521587206463,"percent":61,"absolute":629,"name":"amplitude","payload":{"percent":61,"absolute":629},"from":{"device":{"id":"00e04c0379bb","mac":"00e04c0379bb"}}}
{"type":"input","timestamp":1521587206463,"percent":61,"absolute":629,"name":"amplitude","payload":{"percent":61,"absolute":629},"from":{"device":{"id":"00e04c0379bb","mac":"00e04c0379bb"}}}

This data stream will need to be massaged with the following steps:

  • decode the stream (UTF-8)
  • remove the leading data: , (the remain message will be JSON formatted).
  • load the remain string into a JSON object

Below is some example code, for an app with a slider.

lb_tk_input

#
# lb_input.py - read input from a Cloudbit device
#
import json
import requests

requests.packages.urllib3.disable_warnings()

# Change with your Authorization code and deviceID
authToken = "4f3830b44e1d4b27xxxxxx"
deviceId = "00e04c03xxxx"

littleBitsUrl = "https://api-http.littlebitscloud.cc/devices/" + deviceId + "/input"

headers = {"Authorization": "Bearer " + authToken,"Content-type": "application/json"}
r = requests.get(littleBitsUrl, headers=headers, stream=True)

for line in r.iter_lines():
   strmsg = line.decode("utf-8").strip()
 
   print("Str: ",strmsg)
   if len(strmsg) > 0:
      lb_data = strmsg.split('data:')[1]
      print(lb_data)
      result = json.loads(lb_data)
      thevalue = result['percent']
      lb_bar.set(thevalue)
      thetime = str(datetime.datetime.fromtimestamp(result['timestamp']/1000))
      lb_time['text'] = "Time : " + thetime[:-7]
      root.update()

The Cloud API passed the time stamp as a numeric value with microseconds. The time can be cleaned with:

thetime = str(datetime.datetime.fromtimestamp(result['timestamp']/1000)) 
time_no_micro_seconds = thetime[:-7]

Python Output Value Example

The Cloud API output uses a post method and unlike the input example there is no streaming.

The new output value and a pulse time is sent in the post data parameter. The value is from 0-100, and a pulse time is -1 for sustained, otherwise a millisecond pulse time is used.

Below is some sample code for a Tkinter app:

lb_tk_output

from Tkinter import *
import json
import requests

# update your auth Token and device ID
authToken = "4f3830b44e1d4b27xxxx"
deviceId = "00e04c03xxxxx"

littleBitsUrl = "https://api-http.littlebitscloud.cc/devices/" + deviceId + "/output"

headers = {"Authorization": "Bearer " + authToken, "Content-type": "application/json"}

def setvalue():
	thevalue = lb_value.get()
	thetime = lb_duration.get()
	body = {"percent": thevalue , "duration_ms": thetime}
	thebody = json.dumps(body)
	print body, thebody
	r = requests.post(littleBitsUrl,  data=thebody, headers=headers)
	print r
     

root = Tk()
root.title('littleBits Output')

Label(text = "Value (0-100) :",width=30).grid(row=0,column=0)
lb_value = Entry(root, width= 10)
lb_value.grid(row=0,column=1)
Label(text = "Duration (ms) (constant=-1) :",width=30).grid(row=1,column=0)
lb_duration = Entry(root, width= 10)
lb_duration.grid(row=1,column=1)


Button(root, text=' Set Output ', bg='silver', command=setvalue).grid(row=2,column=1)

root.mainloop()

Final Comments

There are a lot of simple applications and projects that can be created with Python and some littleBits. One of the limitations on the Cloudbit is that it only sends 1 value and receives 1 value. If you are dealing with digital signals it would be possible to do some multiplexing.