Name
Abstract | ||
|---|---|---|
We introduce Picturebook, a large-scale lookup operation to ground language via `snapshots' of our physical world accessed through image search. For each word in a vocabulary, we extract the top-k images from Google image search and feed the images through a convolutional network to extract a word embedding. We introduce a multimodal gating function to fuse our Picturebook embeddings with other word representations. We also introduce Inverse Picturebook, a mechanism to map a Picturebook embedding back into words. We experiment and report results across a wide range of tasks: word similarity, natural language inference, semantic relatedness, sentiment/topic classification, image-sentence ranking and machine translation. We also show that gate activations corresponding to Picturebook embeddings are highly correlated to human judgments of concreteness ratings. |
Year | Venue | Field |
|---|---|---|
2018 | PROCEEDINGS OF THE 56TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL), VOL 1 | Computer science,Ground,Artificial intelligence,Natural language processing,Language understanding |
DocType | Volume | Citations |
Conference | P18-1 | 0 |
PageRank | References | Authors |
0.34 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Kiros, Ryan | 1 | 2265 | 94.80 |
| William Chan | 2 | 357 | 24.67 |
| geoffrey e hinton | 3 | 40435 | 4751.69 |