Skip to content

Instantly share code, notes, and snippets.

@jwalgran
Created February 7, 2019 18:13
Show Gist options
  • Save jwalgran/65c7d50de0fcc3955e4475734156e897 to your computer and use it in GitHub Desktop.
Save jwalgran/65c7d50de0fcc3955e4475734156e897 to your computer and use it in GitHub Desktop.
Dedupe interactive learning session on OAR facility data
> python oar_dedupe_example.py
importing data ...
starting active labeling...
country : cn
name : jiangsu best fashion dress co. ltd
address : no 88 huanxi road, zhutang town, jiangsu
country : cn
name : jiangsu best fashion dress co ltd
address : no88 round west road zhutang town jiangyin city jiangsu
0/10 positive, 0/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished
y
country : bd
name : hams washing and dyeing ltd
address : dakkhin, vangnahati, sreepur 1740, gazipur
country : bd
name : hams washing and dyeing ltd
address : dakkhin, vangnahati, sreepur 1740, gazipur
1/10 positive, 0/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : cn
name : yangzhou jinquan travelling goods co., ltd. (hanjiang)
address : no.16 muyang road hanjiang industrial areas,yangzhou,jiangsu
country : vn
name : youngone hung yen co., ltd.
address : chinh nghia commune kim dong district,hung yen,hung yen
2/10 positive, 0/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : tw
name : hwa meei optical co., ltd. 2nd factory
address : 30, shalun, zhong sha vil., anding dist., tainan, 74555, taiwan
country : ae
name : indigo garments workshop br-2
address : plot no. 104, jurf industrial 1, near ajman car souq, ajman
2/10 positive, 1/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : it
name : calzificio nuovo giodi
address : 10-12 bardella strada, castel goffredo, mantova, 46042
country : it
name : calzificio fap s.p.a.
address : via don p. mazzolari 9, casaloldo,
2/10 positive, 2/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : cn
name : fujian pufeng shoes co., ltd.
address : suanling village, xincuo town, fujian 353011 fuzhou
country : cn
name : zhejiang daqiao shoes co., ltd.
address : dongshan industrial zone, rui'an, zhejiang
2/10 positive, 3/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : lk
name : m i am i c lo th i ng pi ti g ala
address : pelawatta road, pitigala
country : lk
name : miami clothing ltd
address : pelawatte road, pitigala,
2/10 positive, 4/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : rs
name : jeanci serbia d.o.o.
address : tekstilna 57 leskovac 16000,leskovac,leskovac
country : rs
name : jeanci serbia d.o.o.
address : solunskih ratnika bb,16000 leskovac,leskovac,leskovac
3/10 positive, 4/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
u
country : cn
name : bai na shoes industry co., ltd
address : baoyi industrial zone,oubei town,yongjia,zhejiang province
country : cn
name : bai na shoes industry co.,ltd
address : baoyi industrial zone,oubei town,yongjia,wenzhou,zhejiang province
3/10 positive, 4/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : cn
name : far eastern apparal (suzhou) ltd. co
address : #88 tianling road wuzhong development district,suzhou,jiangsu,215128
country : cn
name : far eastern apparel (suzhou) co. ltd
address : 88,tian ling road.,suzhou
4/10 positive, 4/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : pk
name : artistic fabric and garment industries pvt. ltd.
address : plot no. 535471 & 72 sector. 28 korangi industrial area,karachi,sindh
country : pk
name : artistic garment industries unit-8
address : plot # 218, sector. 23, korangi industrial area 74900, karachi
5/10 positive, 4/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : vn
name : jia hsin co., ltd.
address : d1-d6 cau tram industrial park cau tram village long trach commune can duoc dist long an province,can duoc ,long an
country : vn
name : jia hsin co., ltd.
address : lot d1, d2, d3, d4, d5, d6 cau tram industrial park, cau tram village, long trach communecan duoc district, long an
5/10 positive, 5/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : cn
name : zhucheng rili garments co. ltd.
address : south west end of mianzhi street , zhucheng, shandong province , zhucheng, 262200, shandong
country : cn
name : zhucheng yuli garments co. ltd
address : west of xianshi road,bei wai huan,zhucheng city,shandong,china
6/10 positive, 5/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : id
name : pt anugerah cipta mould indonesia
address : jalan millenium raya 3b, blok h-1, millenium industrial estate, desa panongan, tangerang, 15710, banten
country : id
name : pt handal sukses karya
address : jl. millennium raya 3b, blok h-1, millennium industrial estate, desa panongan, tangerang, 15710, banten
6/10 positive, 6/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
u
country : in
name : wearwell india pvt
address : b-43/44 hoisery complex, phase ii, noida, uttar pradesh
country : in
name : wear well india pvt ltd
address : b 43 - 44 hosiery complex phase - ii noida uttar pradesh - 201305
6/10 positive, 6/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : rs
name : olimpias knitting d.o.o.
address : pantelejska 58, 18000 nis, serbia
country : rs
name : tecno stiro
address : pantelejska 58, 18000 nis, serbia
7/10 positive, 6/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : bd
name : the rose dresses ltd
address : zamgara, diakhali,yearpur union, savar
country : bd
name : the rose dresses ltd
address : diakhali, yearpu, jamgora, ashulia 1340, dhaka
7/10 positive, 7/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
u
country : in
name : r.k. & company
address : 1706-7, hsiidc, industrial estate, rai, sonepat, haryana, 131029
country : in
name : r.k. & company
address : b-82 sector - 83 noida uttar pradesh
7/10 positive, 7/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : sv
name : varsity pro ltda de cv (qra)
address : km 36 carretera a santa ana, zona franca american park, poligano "h
country : sv
name : varsity pro ltda de c.v.
address : km 36 carretera a santa ana, zona franca american park, calle canada, poligono h, ciudad arce, na, la libertad
7/10 positive, 8/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
u
country : bd
name : supia cotton
address : north mawna, 1743, sreepur, gazipur, gazipur
country : bd
name : talha fabrics
address : north mawna, 1743, sreepur, gazipur, gazipur
7/10 positive, 8/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : vn
name : unico global vn
address : thuong hamlet, tan an, yen dung,bac giang province,21000,
country : vn
name : unico global vn
address : tan dan commune, yen dung district vietnam
7/10 positive, 9/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : it
name : minoronzoni s.r.l.
address : via enrico fermi 24036, ponte s.pietro
country : it
name : minoronzoni srl
address : via enrico fermi -24036 ponte san pietro bergamo italy,ponte s pietro
7/10 positive, 10/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : bd
name : pretty sweaters ltd
address : holding #222 chayadana, national university, dhaka
country : bd
name : pretty sweaters ltd
address : plot # 222,gacha,, chaydhana, 1704, gazipur
8/10 positive, 10/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
u
country : bd
name : pandora sweaters ltd.
address : gilar chala, master bari, shreepur, gazipur, 1740
country : bd
name : pandora sweaters ltd.
address : master bari, gilar chala, shreepur 1740, gazipur
8/10 positive, 10/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : in
name : aquarelle factory 2
address : 25/1 kempanayakanahalli,jigani hobli,anekal taluk. bangalore,bangalore,karnataka
country : in
name : aquarelle factory 1
address : 1/1 & 51/1 kallabula village,jigani hobli,anekal taluk. bangalore,bangalore,karnataka
9/10 positive, 10/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : kh
name : guang hua gh garment co
address : street prey sor, phum brakar, sangkat prey sor, khan dangkor, phnom penh,
country : kh
name : guang hua.g.h garment co. ltd
address : street prey sor,phum brakar,sangkat prey sor,khan dangkor,phnom penh
9/10 positive, 11/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : id
name : pt pungkook indonesia one
address : jl. raya purwodadi blora km 18 tanjungerjo, wirosari, grobogan 58192
country : id
name : pt. pungkook indonesia one
address : jl. raya pabuaran sarengseng km 3 desa pabuaran kec. pabuaran kab. subang,subang,jawa barat
10/10 positive, 11/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
n
country : cn
name : gaungzhou elegant supreme leather goods co,.ltd
address : jinshi avenue east, no.10, shiling town, 510850, huadu, guangzhou, guangdong
country : cn
name : guangzhou elegant supreme leather goods co,. ltd.
address : no. 10 jinshi avenue east, shiling, town, huadu distirct, guangzhou, city 510850, guangzhou
10/10 positive, 12/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
y
country : vn
name : can sports vietnam co., ltd
address : thuan hoa hamlet, truong mit commune,duong minh chau district,tay ninh province,73000
country : vn
name : espoir vietnam co.,ltd
address : lot f,d1,n3 road,chau son industrial park,chau son ward,,phu ly city,000000,
11/10 positive, 12/10 negative
Do these records refer to the same thing?
(y)es / (n)o / (u)nsure / (f)inished / (p)revious
f
Finished labeling
/Users/jwalgran/.pyenv/versions/dedupe/lib/python3.4/site-packages/dedupe/backport.py:20: UserWarning: NumPy linked against 'Accelerate.framework'. Multiprocessing will be disabled. http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html
warnings.warn("NumPy linked against 'Accelerate.framework'. "
clustering...
# duplicate sets 2398
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment