Jinghong Chen

Sign in Subscribe

Latest

PreFLMR: SoTA Open-sourced Multi-modal Knowledge Retriever from Scaling Up FLMR

[1,087 words, 5-minute read] Three products emerged from our study in scaling up multi-modal late-interaction retrievers: * The Multi-task Multi-modal Knowledge Retrieval benchmark (M2KR) totaling 4.4M training examples for training and comprehensively evaluating knowledge retrievers on question-to-doc, image-to-doc, and question+image-to-doc tasks. * The Pretrained Fine-grained Late-interaction Multi-modal Retriever (PreFLMR)

Deriving Speculative Sampling Intuitively

Retrieval-Augmented Image Synthesis: A Researcher's Guide

3-minute Pitch: Retrieval Guided Contrastive Learning for Hateful Memes Detection

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

Catch up on Speculative Decoding in 5 minutes: a survey for researchers as of December 2023

Estimate LLM inference speed and VRAM usage quickly: with a Llama-7B case study

paper read

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

README. Data Selection (DS) aims to select a given number of samples from a large, unlabeled dataset for training a capable model in a target domain. In the case of training langauge models, practical DS methods need to efficiently select from raw text corpus containing trillions of tokens. This paper,

MultiLoRA explained in 3 minutes: Democratizing LoRA for Better Multi-Task Learning

Papers with Practical Values for Vision-Language Research @NeurIPS 2023 Day 5.

(Vision-Language Researcher) Selected Papers @NeurIPS 2023

Efficient Learning Orals @NeurIPS 2023

Data Contribution Estimation (DCE) for Machine Learning: Tutorial@NeurIPS 2023 in 10-minutes

Highlight Talks @NeurIPS 2023

Highlight Talks @NeurIPS 2023

Articles

Deriving Speculative Sampling Intuitively

[538 words, 2-minute read] A family of lossless LLM inference acceleration techniques has been developed based on speculative sampling (review here). Proposed by Google and Deepmind, speculative sampling is the following three-step procedure: 1. Draft: a small model (draft model, \(p(\cdot|\text{context})\)) quickly generates a K-token draft. 2.

Retrieval-Augmented Image Synthesis: A Researcher's Guide

Catch up on Speculative Decoding in 5 minutes: a survey for researchers as of December 2023

Estimate LLM inference speed and VRAM usage quickly: with a Llama-7B case study

my work

PreFLMR: SoTA Open-sourced Multi-modal Knowledge Retriever from Scaling Up FLMR

[1,087 words, 5-minute read] Three products emerged from our study in scaling up multi-modal late-interaction retrievers: * The Multi-task Multi-modal Knowledge Retrieval benchmark (M2KR) totaling 4.4M training examples for training and comprehensively evaluating knowledge retrievers on question-to-doc, image-to-doc, and question+image-to-doc tasks. * The Pretrained Fine-grained Late-interaction Multi-modal Retriever (PreFLMR)

3-minute Pitch: Retrieval Guided Contrastive Learning for Hateful Memes Detection

3-minute Pitch: Late-Interaction Knowledge Retriever (FLMR) for Visual Question Answering @NeurIPS 2023

3-minute Pitch: Learning from MBR decoding using Direct Preference Optimization