vincchua commited on
Commit
460119b
Β·
verified Β·
1 Parent(s): 9091f2f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -25
README.md CHANGED
@@ -5,8 +5,12 @@ pipeline_tag: text-generation
5
  language:
6
  - en
7
  tags:
 
 
 
8
  - safety
9
  - alignment
 
10
  base_model:
11
  - meta-llama/Llama-3.1-8B-Instruct
12
  - Qwen/Qwen2.5-7B-Instruct
@@ -15,14 +19,15 @@ base_model_relation: adapter
15
  <h1 align="center">HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment</h1>
16
  <p align="center">
17
  <a href="https://arxiv.org/abs/2607.00572"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b.svg?logo=arxiv&logoColor=white" alt="Paper"></a>
18
- <a href="https://huggingface.co/microsoft/HARC"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-yellow.svg" alt="Hugging Face"></a>
19
  <a href="https://github.com/microsoft/HARC"><img src="https://img.shields.io/badge/GitHub-Code-181717.svg?logo=github" alt="GitHub"></a>
20
- <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License: MIT"></a>
21
  </p>
22
 
23
  ---
24
 
25
  ## Model description
 
26
  HARC couples a model's internal *harmfulness* and *refusal* directions at both
27
  prompt-side and response-side token positions, using an additive margin-hinge
28
  loss on cosine projections of the residual stream. The intervention is confined
@@ -31,47 +36,54 @@ layers, which improves robustness to jailbreak attacks while preserving general
31
  capability and avoiding the over-refusal regression typical of broader safety
32
  tuning.
33
 
34
- We provide HARC adapters in this repository. The adapter is applied to attention and MLP projections
35
- and trained with a composite objective:
36
- **(i)** the margin-hinge coupling loss
37
- **(ii)** a KL-divergence retention term anchoring benign outputs to the base model
38
- **(iii)** a cross-entropy term supervising refusal text on harmful prompts.
39
  Training directions are extracted via difference-of-means on contrastive prompt sets
40
  and periodically recomputed with EMA blending. The adapter adds ~1% trainable parameters and leaves the base
41
  architecture unchanged.
42
 
43
-
44
  - **Backbone models:** Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct
 
45
  - **Paper:** [arXiv:2607.00572](https://arxiv.org/abs/2607.00572)
46
  - **Code:** [github.com/microsoft/HARC](https://github.com/microsoft/HARC)
47
 
 
 
 
 
 
 
 
 
 
 
 
 
48
  ## Repository structure
49
- Each model is provided in two forms: the standalone LoRA adapter and a pre-merged
50
- full model (adapter fused into the base weights).
51
 
52
  ```
53
  microsoft/HARC/
54
- β”œβ”€β”€ adapters/
55
- β”‚ β”œβ”€β”€ harc_llama3.1_8b/
56
- β”‚ └── harc_qwen2.5_7b/
57
- └── models/
58
- β”œβ”€β”€ harc_llama3.1_8b/
59
- └── harc_qwen2.5_7b/
60
  ```
61
 
62
  ## How to use
 
63
  Use the base model's standard chat template in both cases.
64
 
65
  ### Option A β€” pre-merged full model (simplest)
66
 
 
 
67
  ```python
68
  from transformers import AutoModelForCausalLM, AutoTokenizer
69
 
70
- repo = "microsoft/HARC"
71
- subfolder = "models/harc_qwen2.5_7b" # or "models/harc_llama3.1_8b"
72
 
73
- tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=subfolder)
74
- model = AutoModelForCausalLM.from_pretrained(repo, subfolder=subfolder, torch_dtype="auto", device_map="auto")
75
 
76
  messages = [{"role": "user", "content": "Hello!"}]
77
  inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
@@ -81,6 +93,8 @@ print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
81
 
82
  ### Option B β€” base model + LoRA adapter (via PEFT)
83
 
 
 
84
  ```python
85
  from transformers import AutoModelForCausalLM, AutoTokenizer
86
  from peft import PeftModel
@@ -98,14 +112,18 @@ hardware requirements match the base model (a 7–8B model in bf16/fp16 fits on
98
  24GB GPU).
99
 
100
  ## Results
101
- <p align="center">
102
- <img src="assets/HARC-res.png" alt="HARC main results on Llama-3.1-8B and Qwen-2.5-7B" width="60%">
103
- </p>
104
 
105
- ## Results
106
- This project is licensed under the MIT License.
 
 
 
 
 
 
107
 
108
  ## Citation
 
109
  ```bibtex
110
  @article{chua2026harc,
111
  title={HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment},
 
5
  language:
6
  - en
7
  tags:
8
+ - lora
9
+ - peft
10
+ - adapter
11
  - safety
12
  - alignment
13
+ - jailbreak-robustness
14
  base_model:
15
  - meta-llama/Llama-3.1-8B-Instruct
16
  - Qwen/Qwen2.5-7B-Instruct
 
19
  <h1 align="center">HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment</h1>
20
  <p align="center">
21
  <a href="https://arxiv.org/abs/2607.00572"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b.svg?logo=arxiv&logoColor=white" alt="Paper"></a>
22
+ <a href="https://huggingface.co/collections/microsoft/harc"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Collection-HARC-ff9d00.svg" alt="Collection"></a>
23
  <a href="https://github.com/microsoft/HARC"><img src="https://img.shields.io/badge/GitHub-Code-181717.svg?logo=github" alt="GitHub"></a>
24
+ <a href="https://huggingface.co/microsoft/HARC/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License: MIT"></a>
25
  </p>
26
 
27
  ---
28
 
29
  ## Model description
30
+
31
  HARC couples a model's internal *harmfulness* and *refusal* directions at both
32
  prompt-side and response-side token positions, using an additive margin-hinge
33
  loss on cosine projections of the residual stream. The intervention is confined
 
36
  capability and avoiding the over-refusal regression typical of broader safety
37
  tuning.
38
 
39
+ **This repository contains the HARC LoRA adapters.** The adapter is applied to attention and MLP projections
40
+ and trained with a composite objective: **(i)** the margin-hinge coupling loss **(ii)** a KL-divergence retention term anchoring benign outputs to the base model **(iii)** a cross-entropy term supervising refusal text on harmful prompts.
 
 
 
41
  Training directions are extracted via difference-of-means on contrastive prompt sets
42
  and periodically recomputed with EMA blending. The adapter adds ~1% trainable parameters and leaves the base
43
  architecture unchanged.
44
 
 
45
  - **Backbone models:** Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct
46
+ - **Collection:** [HARC Collection](https://huggingface.co/collections/microsoft/harc)
47
  - **Paper:** [arXiv:2607.00572](https://arxiv.org/abs/2607.00572)
48
  - **Code:** [github.com/microsoft/HARC](https://github.com/microsoft/HARC)
49
 
50
+ ## The HARC collection
51
+
52
+ | Repo | Contents | License |
53
+ |---|---|---|
54
+ | **microsoft/HARC** (this repo) | LoRA adapters for both backbones | MIT |
55
+ | [microsoft/HARC-Llama-3.1-8B-Instruct](https://huggingface.co/microsoft/HARC-Llama-3.1-8B-Instruct) | Merged full model | Llama 3.1 Community License |
56
+ | [microsoft/HARC-Qwen2.5-7B-Instruct](https://huggingface.co/microsoft/HARC-Qwen2.5-7B-Instruct) | Merged full model | Apache-2.0 |
57
+
58
+ Use this repo if you want the lightweight adapters to load on top of your own
59
+ copy of the base model; use the merged-model repos if you want a single
60
+ ready-to-run checkpoint.
61
+
62
  ## Repository structure
 
 
63
 
64
  ```
65
  microsoft/HARC/
66
+ └── adapters/
67
+ β”œβ”€β”€ harc_llama3.1_8b/ # base = Llama-3.1-8B-Instruct
68
+ └── harc_qwen2.5_7b/ # base = Qwen2.5-7B-Instruct
 
 
 
69
  ```
70
 
71
  ## How to use
72
+
73
  Use the base model's standard chat template in both cases.
74
 
75
  ### Option A β€” pre-merged full model (simplest)
76
 
77
+ Loads directly from the merged-model repo; no base download or PEFT required.
78
+
79
  ```python
80
  from transformers import AutoModelForCausalLM, AutoTokenizer
81
 
82
+ # pick the merged model you want
83
+ repo = "microsoft/HARC-Qwen2.5-7B-Instruct" # or "microsoft/HARC-Llama-3.1-8B-Instruct"
84
 
85
+ tokenizer = AutoTokenizer.from_pretrained(repo)
86
+ model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
87
 
88
  messages = [{"role": "user", "content": "Hello!"}]
89
  inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
 
93
 
94
  ### Option B β€” base model + LoRA adapter (via PEFT)
95
 
96
+ Load the base model, then attach the adapter from this repo with the matching `subfolder`.
97
+
98
  ```python
99
  from transformers import AutoModelForCausalLM, AutoTokenizer
100
  from peft import PeftModel
 
112
  24GB GPU).
113
 
114
  ## Results
 
 
 
115
 
116
+ ![HARC main results on Llama-3.1-8B and Qwen-2.5-7B](https://huggingface.co/microsoft/HARC/resolve/main/assets/HARC-res.png)
117
+
118
+ ## License
119
+
120
+ The LoRA adapters in this repository are released under the MIT License. The
121
+ merged full models are distributed in separate repositories under their base
122
+ model's license: the Llama variant under the Meta Llama 3.1 Community License,
123
+ and the Qwen variant under Apache-2.0.
124
 
125
  ## Citation
126
+
127
  ```bibtex
128
  @article{chua2026harc,
129
  title={HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment},