Jinghong Chen

Sign in Subscribe

Latest

The Six Elements of NLP Experiments. Part 1: Datasets

Experiments are the foundations of Natural Languge Processing (NLP). Just like experimental sciences, we test our proposed methods and verify our theories with experiments. Describing, planning, and conducting experiments are therefore the core skills of every NLP researcher. This article concerns describing NLP experiments. There are six basic elements that

Telling AI from Human: DeepFake Texts in the era of Large Language Models. Tutorial Notes @NAACL 2024

Telling AI from Human: DeepFake Texts in the era of Large Language Models. Tutorial Notes @NAACL 2024

Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA)

Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA)

PreFLMR: SoTA Open-sourced Multi-modal Knowledge Retriever from Scaling Up FLMR

Deriving Speculative Sampling Intuitively

Retrieval-Augmented Image Synthesis: A Researcher's Guide

3-minute Pitch: Retrieval Guided Contrastive Learning for Hateful Memes Detection

Articles

Deriving Speculative Sampling Intuitively

[538 words, 2-minute read] A family of lossless LLM inference acceleration techniques has been developed based on speculative sampling (review here). Proposed by Google and Deepmind, speculative sampling is the following three-step procedure: 1. Draft: a small model (draft model, \(p(\cdot|\text{context})\)) quickly generates a K-token draft. 2.

Retrieval-Augmented Image Synthesis: A Researcher's Guide

Catch up on Speculative Decoding in 5 minutes: a survey for researchers as of December 2023

Estimate LLM inference speed and VRAM usage quickly: with a Llama-7B case study

Papers with Practical Values for Vision-Language Research @NeurIPS 2023 Day 5.

(Vision-Language Researcher) Selected Papers @NeurIPS 2023

Efficient Learning Orals @NeurIPS 2023

my work

Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA)

Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA)

[4-minute read] TL;DR. Non-autoregressive (NAR) models generate texts much faster than auto-regresssive (AR) models. However, we find previous NAR approaches, largely developed for Machine Translation, fail harshly when faced with Task-Oriented Dialogue and Data-to-Text. Our NAACL 2024 paper introduces Control-DAG, a constrained decoding algorithm that uses Weighted Finite State

PreFLMR: SoTA Open-sourced Multi-modal Knowledge Retriever from Scaling Up FLMR

3-minute Pitch: Retrieval Guided Contrastive Learning for Hateful Memes Detection

3-minute Pitch: Late-Interaction Knowledge Retriever (FLMR) for Visual Question Answering @NeurIPS 2023

3-minute Pitch: Learning from MBR decoding using Direct Preference Optimization

paper read

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

README. Data Selection (DS) aims to select a given number of samples from a large, unlabeled dataset for training a capable model in a target domain. In the case of training langauge models, practical DS methods need to efficiently select from raw text corpus containing trillions of tokens. This paper,

MultiLoRA explained in 3 minutes: Democratizing LoRA for Better Multi-Task Learning

Papers with Practical Values for Vision-Language Research @NeurIPS 2023 Day 5.