scientific-thinking-scholar-evaluation

affaan-m ｜ 資料與分析 ｜ 原文 ｜ MIT

用固定評分標準評估論文和研究計畫的品質

---
name: scientific-thinking-scholar-evaluation
description: 針對論文、研究計畫、文獻回顧、方法段落、證據品質、引用佐證和研究寫作回饋，做結構化的學術作品評估。用可重複使用的評分標準評估學術或科學作品（論文、計畫、方法段落或證據品質）時使用。
metadata:
  origin: community
---

# 學術評估

用可重複使用的評分標準評估學術或科學作品時，使用這個 skill。

## 何時使用

- Review 研究論文、研究計畫、論文章節或文獻回顧。
- 檢查主張有沒有被引用的證據支持。
- 評估方法論、研究設計、分析或限制。
- 比較兩篇以上論文的品質或相關性。
- 產出結構化的修改回饋。

## 評估範圍

先確認對象是哪一種：

- 實證研究論文
- 理論論文
- 技術報告
- 系統性或敘述性文獻回顧
- 研究計畫
- 論文或博士論文章節
- 會議摘要或短論文

再選範圍：

- **comprehensive（全面）**：所有評分維度
- **targeted（針對性）**：一兩個維度，例如方法或引用
- **comparative（比較性）**：用同一份評分標準替多篇作品排名

## 評分標準

每個適用的維度，從 1 到 5 打分：

- 5：優秀；清楚、嚴謹，可以直接發表
- 4：良好；需要小幅改進
- 3：尚可；有明顯缺口但還能用
- 2：薄弱；需要大幅修改
- 1：差；有重大的效度或清晰度問題

不適用的維度用 `N/A`。

### 1. 問題與研究問題

- 問題清楚而且具體嗎？
- 貢獻有意義嗎？
- 範圍和假設有明確寫出嗎？
- 研究問題跟所宣稱的貢獻相符嗎？

### 2. 文獻與脈絡

- 有涵蓋相關的前人研究嗎？
- 作品是綜合整理，還是只是列出來源？
- 缺口有被正確指出嗎？
- 近期文獻和奠基性文獻有取得平衡嗎？

### 3. 方法論

- 方法能回答研究問題嗎？
- 設計選擇有被說明理由嗎？
- 變數、資料集、受試者或材料描述得清楚嗎？
- 其他研究者能重現這項工作嗎？
- 有承認倫理與實務上的限制嗎？

### 4. 資料與證據

- 資料來源可信而且合適嗎？
- 樣本數或語料涵蓋範圍足夠嗎？
- 納入、排除和前處理的決定有記錄嗎？
- 有討論缺失資料和偏誤風險嗎？

### 5. 分析

- 統計、質性或計算方法合適嗎？
- Baseline 和對照組公平嗎？
- 需要時有包含不確定性、敏感度或穩健性檢查嗎？
- 有考慮其他解釋嗎？

### 6. 結果與詮釋

- 結果呈現得清楚嗎？
- 主張有沒有維持在證據範圍內？
- 圖、表和指標看得懂嗎？
- 負面或虛無結果有被誠實處理嗎？

### 7. 限制與效度威脅

- 限制是具體的，還是籠統的？
- 有處理內部、外部、建構和結論效度的風險嗎？
- 論文有區分推測和已驗證的結果嗎？

### 8. 寫作與結構

- 論述容易跟上嗎？
- 各段落是圍繞研究問題組織的嗎？
- 定義和符號清楚嗎？
- 語氣精確而且有學術味嗎？

### 9. 引用

- 被引用的論文真的支持附在它們身上的主張嗎？
- 有盡可能使用原始來源嗎？
- 回顧文獻有標示為回顧嗎？
- 預印本有標示為預印本嗎？
- 引用的 metadata 和連結正確嗎？

## Review 流程

1. 讀摘要、前言、圖表和結論，掌握所宣稱的貢獻。
2. 讀方法和結果，評估證據品質。
3. 把最強的主張對照被引用的來源檢查。
4. 為每個適用的維度打分。
5. 把關鍵的阻礙和修改建議分開。
6. 最後給出具體的下一步修改。

## 輸出範本

```markdown
# Scholar Evaluation: <Artifact>

## Overall Assessment

- Overall score: <1-5 or N/A>
- Confidence: <high | medium | low>
- Summary: <3-5 sentences>

## Dimension Scores

| Dimension | Score | Evidence | Revision priority |
| --- | ---: | --- | --- |
| Problem and question |  |  |  |
| Literature and context |  |  |  |
| Methodology |  |  |  |
| Data and evidence |  |  |  |
| Analysis |  |  |  |
| Results and interpretation |  |  |  |
| Limitations |  |  |  |
| Writing and structure |  |  |  |
| Citations |  |  |  |

## Critical Issues

## Recommended Revisions

## Evidence Checks Needed
```

## 常見陷阱

- 不要拿分數取代具體的回饋。
- 不要因為論文沒涵蓋它範圍之外的維度就扣分。
- 不要把引用次數、發表場域或作者名氣當成品質的證明。
- 不要因為無根據的主張出現在摘要裡，就照單全收。

---
name: scientific-thinking-scholar-evaluation
description: Structured scholarly-work evaluation for papers, proposals, literature reviews, methods sections, evidence quality, citation support, and research-writing feedback. Use when evaluating academic or scientific work — papers, proposals, methods sections, or evidence quality — against a repeatable rubric.
metadata:
  origin: community
---

# Scholar Evaluation

Use this skill to evaluate academic or scientific work with a repeatable rubric.

## When to Use

- Reviewing a research paper, proposal, thesis chapter, or literature review.
- Checking whether claims are supported by cited evidence.
- Evaluating methodology, study design, analysis, or limitations.
- Comparing two or more papers for quality or relevance.
- Producing structured feedback for revision.

## Evaluation Scope

Start by identifying the artifact:

- empirical research paper
- theoretical paper
- technical report
- systematic or narrative literature review
- research proposal
- thesis or dissertation chapter
- conference abstract or short paper

Then choose scope:

- **comprehensive**: all rubric dimensions
- **targeted**: one or two dimensions, such as method or citations
- **comparative**: rank multiple works against the same rubric

## Rubric

Score each applicable dimension from 1 to 5:

- 5: excellent; clear, rigorous, and publication-ready
- 4: good; minor improvements needed
- 3: adequate; meaningful gaps but usable
- 2: weak; substantial revision needed
- 1: poor; major validity or clarity problems

Use `N/A` for dimensions that do not apply.

### 1. Problem and Research Question

- Is the problem clear and specific?
- Is the contribution meaningful?
- Are scope and assumptions explicit?
- Does the question match the claimed contribution?

### 2. Literature and Context

- Is relevant prior work covered?
- Does the work synthesize rather than merely list sources?
- Are gaps accurately identified?
- Are recent and foundational sources balanced?

### 3. Methodology

- Does the method answer the research question?
- Are design choices justified?
- Are variables, datasets, participants, or materials described clearly?
- Could another researcher reproduce the work?
- Are ethical and practical constraints acknowledged?

### 4. Data and Evidence

- Are data sources credible and appropriate?
- Is sample size or corpus coverage adequate?
- Are inclusion, exclusion, and preprocessing decisions documented?
- Are missing data and bias risks discussed?

### 5. Analysis

- Are statistical, qualitative, or computational methods appropriate?
- Are baselines and controls fair?
- Are uncertainty, sensitivity, or robustness checks included when needed?
- Are alternative explanations considered?

### 6. Results and Interpretation

- Are results clearly presented?
- Do claims stay within the evidence?
- Are figures, tables, and metrics understandable?
- Are negative or null results handled honestly?

### 7. Limitations and Threats to Validity

- Are limitations specific rather than generic?
- Are internal, external, construct, and conclusion-validity risks addressed?
- Does the paper distinguish speculation from demonstrated results?

### 8. Writing and Structure

- Is the argument easy to follow?
- Are sections organized around the research question?
- Are definitions and notation clear?
- Is the tone precise and scholarly?

### 9. Citations

- Do cited papers support the claims attached to them?
- Are primary sources used where possible?
- Are reviews labeled as reviews?
- Are preprints labeled as preprints?
- Are citation metadata and links correct?

## Review Process

1. Read the abstract, introduction, figures, and conclusion for claimed
   contribution.
2. Read methods and results for evidence quality.
3. Check the strongest claims against cited sources.
4. Score each applicable dimension.
5. Separate critical blockers from revision suggestions.
6. End with concrete next edits.

## Output Template

```markdown
# Scholar Evaluation: <Artifact>

## Overall Assessment

- Overall score: <1-5 or N/A>
- Confidence: <high | medium | low>
- Summary: <3-5 sentences>

## Dimension Scores

| Dimension | Score | Evidence | Revision priority |
| --- | ---: | --- | --- |
| Problem and question |  |  |  |
| Literature and context |  |  |  |
| Methodology |  |  |  |
| Data and evidence |  |  |  |
| Analysis |  |  |  |
| Results and interpretation |  |  |  |
| Limitations |  |  |  |
| Writing and structure |  |  |  |
| Citations |  |  |  |

## Critical Issues

## Recommended Revisions

## Evidence Checks Needed
```

## Pitfalls

- Do not use the score as a substitute for concrete feedback.
- Do not penalize a paper for omitting a dimension outside its scope.
- Do not treat citation count, venue, or author reputation as proof of quality.
- Do not accept unsupported claims just because they appear in the abstract.