Jinghong Chen

Sign in Subscribe

Latest

[Quick Pitch] On Extending Direct Preference Optimization to Accommodate Ties @NeurIPS 2025

Direct Preference Optimization (DPO) trains language models on pairs of preferred and dispreferred responses, \(y_w \succ y_l\). But not all pairs have a clear winner. For pairs without clear preference - i.e., ties - a common approach is to simply discard them (e.g., Llama3 and Qwen2)

Generative AI should amplify human creativity, not replace it - my positions in 2025

The Six Elements of NLP Experiments. Part 1: Datasets

Telling AI from Human: DeepFake Texts in the era of Large Language Models. Tutorial Notes @NAACL 2024

Telling AI from Human: DeepFake Texts in the era of Large Language Models. Tutorial Notes @NAACL 2024

[Quick Pitch] Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA) @NAACL 2024

[Quick Pitch] Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA) @NAACL 2024

[Quick Pitch] PreFLMR: SoTA Open-sourced Multi-modal Knowledge Retriever from Scaling Up FLMR

[Review] Deriving Speculative Sampling Intuitively

Articles

[Review] Deriving Speculative Sampling Intuitively

[538 words, 2-minute read] A family of lossless LLM inference acceleration techniques has been developed based on speculative sampling (review here). Proposed by Google and Deepmind, speculative sampling is the following three-step procedure: 1. Draft: a small model (draft model, \(p(\cdot|\text{context})\)) quickly generates a K-token draft. 2.

[Review] Retrieval-Augmented Image Synthesis: A Researcher's Guide

[Review] Catch up on Speculative Decoding in 5 minutes: a survey for researchers as of December 2023

Estimate LLM inference speed and VRAM usage quickly: with a Llama-7B case study

Papers with Practical Values for Vision-Language Research @NeurIPS 2023 Day 5.

(Vision-Language Researcher) Selected Papers @NeurIPS 2023

Efficient Learning Orals @NeurIPS 2023

my work

[Quick Pitch] On Extending Direct Preference Optimization to Accommodate Ties @NeurIPS 2025

Direct Preference Optimization (DPO) trains language models on pairs of preferred and dispreferred responses, \(y_w \succ y_l\). But not all pairs have a clear winner. For pairs without clear preference - i.e., ties - a common approach is to simply discard them (e.g., Llama3 and Qwen2)

[Quick Pitch] Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA) @NAACL 2024

[Quick Pitch] Control-DAG: Constraining Non-Autoregressive Text Generation with Weighted Finite State Automata (WFSA) @NAACL 2024

[Quick Pitch] PreFLMR: SoTA Open-sourced Multi-modal Knowledge Retriever from Scaling Up FLMR

[Quick Pitch] Retrieval Guided Contrastive Learning for Hateful Memes Detection

[Quick Pitch] Late-Interaction Knowledge Retriever (FLMR) for Visual Question Answering @NeurIPS 2023

[Quick Pitch] Learning from MBR decoding using Direct Preference Optimization

paper read

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

[Paper Express] Data Selection for Language Models via Importance Resampling (DSIR)

README. Data Selection (DS) aims to select a given number of samples from a large, unlabeled dataset for training a capable model in a target domain. In the case of training langauge models, practical DS methods need to efficiently select from raw text corpus containing trillions of tokens. This paper,

[Review] MultiLoRA explained in 3 minutes: Democratizing LoRA for Better Multi-Task Learning

Papers with Practical Values for Vision-Language Research @NeurIPS 2023 Day 5.