์—…๋ฐ์ดํŠธ๋กœ ๋Œ์•„๊ฐ€๊ธฐ
New releaseSep 15, 2026

giskard-oss giskard-checks/v1.0.4

๐Ÿข LLM ์—์ด์ „ํŠธ์šฉ ์˜คํ”ˆ์†Œ์Šค ํ‰๊ฐ€ ๋ฐ ํ…Œ์ŠคํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ

๊ณต์œ 

giskardlogo giskardlogo

์—์ด์ „ํŠธ ์‹œ์Šคํ…œ์„ ์œ„ํ•œ ํ‰๊ฐ€, ๋ ˆ๋“œ ํŒ€ ํ™œ๋™ ๋ฐ ํ…Œ์ŠคํŠธ ์ƒ์„ฑ

๋ชจ๋“ˆํ˜•, ๊ฒฝ๋Ÿ‰, ๋™์  ๋ฐ ๋น„๋™๊ธฐ ์šฐ์„ 

GitHub release License Downloads CI Giskard on Discord

๋ฌธ์„œ โ€ข ์›น์‚ฌ์ดํŠธ โ€ข ์ปค๋ฎค๋‹ˆํ‹ฐ


[!IMPORTANT] Giskard v3๋Š” AI ์—์ด์ „ํŠธ์˜ ๋™์ ์ด๊ณ  ๋‹ค์ค‘ ํ„ด ํ…Œ์ŠคํŠธ๋ฅผ ์œ„ํ•ด ์„ค๊ณ„๋œ ์™„์ „ํžˆ ์ƒˆ๋กœ ์ž‘์„ฑ๋œ ๋ฒ„์ „์ž…๋‹ˆ๋‹ค. ์ด ๋ฆด๋ฆฌ์Šค๋Š” ๋” ๋‚˜์€ ํšจ์œจ์„ฑ์„ ์œ„ํ•ด ๋ฌด๊ฑฐ์šด ์˜์กด์„ฑ์„ ์ œ๊ฑฐํ•˜๋Š” ๋™์‹œ์— ๋” ๊ฐ•๋ ฅํ•œ AI ์ทจ์•ฝ์  ์Šค์บ๋„ˆ์™€ ํ–ฅ์ƒ๋œ RAG ํ‰๊ฐ€๋ฅผ ๋„์ž…ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋‘ ๊ธฐ๋Šฅ ๋ชจ๋‘ ์ด์ œ v2์— ๋Œ€ํ•œ ์˜์กด์„ฑ ์—†์ด giskard-scan์— ๊ธฐ๋ณธ์œผ๋กœ ์ œ๊ณต๋ฉ๋‹ˆ๋‹ค. ํ‘œ ํ˜•์‹/ML ๋ชจ๋ธ์„ ์œ„ํ•œ ๋ ˆ๊ฑฐ์‹œ ์Šค์บ”๋งŒ์ด ์—ฌ์ „ํžˆ v2 ์ „์šฉ์ž…๋‹ˆ๋‹ค. Giskard v2๋Š” ๊ณ„์† ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์ง€๋งŒ ๋” ์ด์ƒ ์ ๊ทน์ ์œผ๋กœ ์œ ์ง€ ๊ด€๋ฆฌ๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ์ง„ํ–‰ ์ƒํ™ฉ ํŒ”๋กœ์šฐ โ†’ v3 ๋ฐœํ‘œ ์ฝ๊ธฐ ยท ๋กœ๋“œ๋งต

์„ค์น˜

pip install giskard           # checks (+ agents, llm, core)
pip install "giskard[scan]"   # + vulnerability / quality scan
pip install "giskard[openai]" # provider SDK for LLM judges / generators

Python 3.12+๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.

์ถ”๊ฐ€ ํ•ญ๋ชฉ์ถ”๊ฐ€๋˜๋Š” ๋‚ด์šฉ
(์—†์Œ)giskard-checks ๋ฐ ์ข…์†์„ฑ
scangiskard-scan
openai / anthropic / โ€ฆ์ œ๊ณต์ž SDK (pyproject.toml์˜ ์„ ํƒ์  ์ข…์†์„ฑ ์ฐธ์กฐ)

ํ…”๋ ˆ๋ฉ”ํŠธ๋ฆฌ: giskard-core๋ฅผ ํ†ตํ•œ ์„ ํƒ์  ์ง‘๊ณ„ ๋ถ„์„. ํ”„๋กฌํ”„ํŠธ๋‚˜ ์ถœ๋ ฅ์€ ์ „์†ก๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. export DO_NOT_TRACK=1 ๋˜๋Š” export GISKARD_TELEMETRY_DISABLED=1๋กœ ์˜ตํŠธ์•„์›ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค(๋˜๋Š” ์ž‘์—… ๋””๋ ‰ํ„ฐ๋ฆฌ์˜ .env ํŒŒ์ผ์— ๋™์ผํ•œ ํ‚ค ์„ค์ •). ~/.giskard/id ์ƒ์„ฑ์„ ๊ฑด๋„ˆ๋›ฐ๋ ค๋ฉด import ์ „์— ์„ค์ •ํ•˜์„ธ์š”. ๋‚˜์ค‘์— ์„ค์ •ํ•ด๋„ ์ถ”๊ฐ€ ์ „์†ก์€ ์ค‘๋‹จ๋ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ: giskard-core README.


Giskard๋Š” ์—์ด์ „ํŠธ ์‹œ์Šคํ…œ์„ ํ…Œ์ŠคํŠธํ•˜๊ณ  ํ‰๊ฐ€ํ•˜๊ธฐ ์œ„ํ•œ ์˜คํ”ˆ์†Œ์Šค Python ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์ž…๋‹ˆ๋‹ค. v3 ์•„ํ‚คํ…์ฒ˜๋Š” ๊ฐ๊ฐ ํ•„์š”ํ•œ ์ข…์†์„ฑ๋งŒ ํฌํ•จํ•˜๋Š”, ๋ชฉ์ ์— ์ง‘์ค‘ํ•œ ๋ชจ๋“ˆํ˜• ํŒจํ‚ค์ง€ ์ง‘ํ•ฉ์œผ๋กœ, LLM, ๋ธ”๋ž™๋ฐ•์Šค ์—์ด์ „ํŠธ, ๋‹ค๋‹จ๊ณ„ ํŒŒ์ดํ”„๋ผ์ธ ๋“ฑ ๋ฌด์—‡์ด๋“  ๊ฐ์Œ€ ์ˆ˜ ์žˆ๋„๋ก ์ฒ˜์Œ๋ถ€ํ„ฐ ๊ตฌ์ถ•๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

์ƒํƒœํŒจํ‚ค์ง€์„ค๋ช…
โœ… ์•ˆ์ •giskard-checksํ…Œ์ŠคํŠธ ๋ฐ ํ‰๊ฐ€ โ€” ์‹œ๋‚˜๋ฆฌ์˜ค API, ๋‚ด์žฅ ๊ฒ€์‚ฌ, LLM-as-judge
โœ… ์•ˆ์ •giskard-scan์—์ด์ „ํŠธ ์ทจ์•ฝ์  ์Šค์บ๋„ˆ + RAG/ํ’ˆ์งˆ ํ‰๊ฐ€ โ€” ๋ ˆ๋“œ ํŒ€์ž‰, ํ”„๋กฌํ”„ํŠธ ์ธ์ ์…˜, ํƒˆ์˜ฅ ๋ฐ ์œ ํ•ด ์ฝ˜ํ…์ธ (v2 Scan์˜ ํ›„์†์ธ vulnerability_scan), ๊ทธ๋ฆฌ๊ณ  ์ง€์‹ ๊ธฐ๋ฐ˜ ํ’ˆ์งˆ ํ‰๊ฐ€(v2 RAGET์˜ ํ›„์†์ธ quality_scan)

์ด๋“ค์€ ์„ธ ๊ฐ€์ง€ ๊ธฐ๋ฐ˜ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ โ€” giskard-core(๊ณต์œ  ์œ ํ‹ธ๋ฆฌํ‹ฐ ๋ฐ ํ…”๋ ˆ๋ฉ”ํŠธ๋ฆฌ), giskard-llm(์ œ๊ณต์ž ๋…๋ฆฝ์  LLM ๋ผ์šฐํŒ…), giskard-agents(์—์ด์ „ํŠธ ๋ฐ ์›Œํฌํ”Œ๋กœ ์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜) โ€” ์œ„์— ๊ตฌ์ถ•๋˜๋ฉฐ, ์ด๋“ค์€ ์ž๋™์œผ๋กœ ๊ฐ€์ ธ์™€์ง€๊ณ  ์ง์ ‘ ์‚ฌ์šฉ๋˜๋Š” ๊ฒฝ์šฐ๋Š” ๋“œ๋ญ…๋‹ˆ๋‹ค.

Giskard Checks โ€” ์—์ด์ „ํŠธ ํ…Œ์ŠคํŠธ๋ฅผ ์œ„ํ•œ ํ‰๊ฐ€ ์ƒ์„ฑ ๋ฐ ์ ์šฉ

pip install giskard-checks

**Giskard Checks**๋Š” LLM ๊ธฐ๋ฐ˜ ์‹œ์Šคํ…œ์„ ํ…Œ์ŠคํŠธํ•˜๋Š” ํ‰๊ฐ€(eval)๋ฅผ ๋งŒ๋“ค๊ธฐ ์œ„ํ•œ ๊ฒฝ๋Ÿ‰ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋กœ, ๋‹จ์ˆœํ•œ ๊ฒ€์ฆ(assertion)๋ถ€ํ„ฐ LLM-as-judge ํ‰๊ฐ€๊นŒ์ง€ ์ง€์›ํ•ฉ๋‹ˆ๋‹ค. ์ „ํ†ต์ ์ธ ๋‹จ์œ„ ํ…Œ์ŠคํŠธ์™€ ๋‹ฌ๋ฆฌ, eval์€ ๋™์ผํ•œ ์ž…๋ ฅ์ด ์„œ๋กœ ๋‹ค๋ฅธ ์œ ํšจํ•œ ์‘๋‹ต์„ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ๋Š” ๋น„๊ฒฐ์ •์  ์ถœ๋ ฅ์„ ์œ„ํ•ด ์„ค๊ณ„๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

Giskard Checks๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ:

  • ํšŒ๊ท€ ๊ฐ์ง€ โ€” ๋ณ€๊ฒฝ ํ›„์—๋„ ์‹œ์Šคํ…œ์ด ์—ฌ์ „ํžˆ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ๋™์ž‘ํ•˜๋Š”์ง€ ํ™•์ธ
  • RAG ํ’ˆ์งˆ ๊ฒ€์ฆ โ€” ๋‹ต๋ณ€์ด ๊ฒ€์ƒ‰๋œ ์ปจํ…์ŠคํŠธ์— ๊ทผ๊ฑฐํ•˜๋Š”์ง€ ํ™•์ธ
  • ์•ˆ์ „ ๊ทœ์น™ ์ ์šฉ โ€” ์ถœ๋ ฅ์ด ์ฝ˜ํ…์ธ  ์ •์ฑ…์„ ์ค€์ˆ˜ํ•˜๋Š”์ง€ ๋ณด์žฅ
  • ๋ฉ€ํ‹ฐํ„ด ์—์ด์ „ํŠธ ํ‰๊ฐ€ โ€” ๋‹จ์ผ ๊ตํ™˜๋ฟ ์•„๋‹ˆ๋ผ ์ „์ฒด ๋Œ€ํ™” ํ…Œ์ŠคํŠธ

๋‚ด์žฅ eval์—๋Š” ๋ฌธ์ž์—ด ๋งค์นญ, ๋น„๊ต, ์ •๊ทœ์‹, ์˜๋ฏธ์  ์œ ์‚ฌ๋„, LLM-as-judge ๊ฒ€์‚ฌ(Groundedness, Conformity, LLMJudge)๊ฐ€ ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.

๊ฐœ๋…

  • Target โ€” ํ…Œ์ŠคํŠธ ๋Œ€์ƒ ์‹œ์Šคํ…œ: ์ž„์˜์˜ ๋™๊ธฐ/๋น„๋™๊ธฐ ํ˜ธ์ถœ ๊ฐ€๋Šฅ ๊ฐ์ฒด (inputs) -> outputs (์„ ํƒ์ ์œผ๋กœ trace ํฌํ•จ)
  • Scenario โ€” ํ•˜๋‚˜์˜ eval: ์ƒํ˜ธ์ž‘์šฉ + ๊ฒ€์‚ฌ
  • Check โ€” ํŠธ๋ ˆ์ด์Šค์— ๋Œ€ํ•œ ๊ฒ€์ฆ(assertion) ๋˜๋Š” LLM ํŒ์ •
  • Suite โ€” ํ•จ๊ป˜ ์‹คํ–‰๋˜๋Š” ์—ฌ๋Ÿฌ ์‹œ๋‚˜๋ฆฌ์˜ค

giskard.agents.Generator๋Š” ์›Œํฌํ”Œ๋กœ/ํŒ์ •์„ ์œ„ํ•œ LLM ํด๋ผ์ด์–ธํŠธ๋กœ, ์‚ฌ์šฉ์ž ๋ฉ”์‹œ์ง€๋ฅผ ํ•ฉ์„ฑํ•˜๋Š” giskard.checks ์ž…๋ ฅ ์ƒ์„ฑ๊ธฐ(LLMGenerator)์™€๋Š” ๋‹ค๋ฆ…๋‹ˆ๋‹ค.

๋น ๋ฅธ ์‹œ์ž‘

์‹คํ–‰ ์ค‘์ธ ์ด๋ฒคํŠธ ๋ฃจํ”„๊ฐ€ ์—†๋Š” ์Šคํฌ๋ฆฝํŠธ์—์„œ:

from giskard.checks import Scenario, Groundedness


def get_answer(inputs: str) -> str:
    return "Paris"  # replace with your model / agent


scenario = (
    Scenario("test_france_capital")
    .interact(inputs="What is the capital of France?", outputs=get_answer)
    .check(
        Groundedness(
            name="answer is grounded",
            context="France is in Western Europe. Its capital is Paris.",
        )
    )
)
result = scenario.run_sync()
result.print_report()

๋…ธํŠธ๋ถ ์…€์ด๋‚˜ ๋น„๋™๊ธฐ ํ•จ์ˆ˜ ๋‚ด๋ถ€์—์„œ๋Š” ๋Œ€์‹  ์‹œ๋‚˜๋ฆฌ์˜ค๋ฅผ awaitํ•˜์„ธ์š”:

result = await scenario.run()
result.print_report()

Groundedness๋Š” LLM ํŒ์ •๊ธฐ์ž…๋‹ˆ๋‹ค โ€” ์ œ๊ณต์ž ์ถ”๊ฐ€ ํŒจํ‚ค์ง€๋ฅผ ์„ค์น˜ํ•˜๊ณ (์˜ˆ: pip install "giskard[openai]") ํ•ด๋‹น API ํ‚ค๋ฅผ ์„ค์ •ํ•˜์„ธ์š”. ๊ธฐ๋ณธ ๋ชจ๋ธ: openai/gpt-4o-mini.

Suites, LLMJudge, ๋ฉ€ํ‹ฐํ„ด ์‹œ๋‚˜๋ฆฌ์˜ค ๋“ฑ์— ๋Œ€ํ•œ ๋‚ด์šฉ์€ ์ „์ฒด ๋ฌธ์„œ๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.


Giskard Scan โ€” AI ์—์ด์ „ํŠธ์šฉ ์ทจ์•ฝ์  ์Šค์บ๋„ˆ

pip install "giskard[scan]"   # or: pip install giskard-scan

Giskard Scan์€ ์—์ด์ „ํ‹ฑ ์‹œ์Šคํ…œ์„ ์œ„ํ•œ ๋ ˆ๋“œํŒ€ ๋ฐ ์ทจ์•ฝ์  ์Šค์บ๋‹ ๊ณ„์ธต์ž…๋‹ˆ๋‹ค. ์—์ด์ „ํŠธ์— ๋Œ€ํ•œ ํ‰์ดํ•œ ์–ธ์–ด ์„ค๋ช…์œผ๋กœ๋ถ€ํ„ฐ ํ”„๋กฌํ”„ํŠธ ์ธ์ ์…˜, ์œ ํ•ด ์ฝ˜ํ…์ธ , ๊ณ ์ •๊ด€๋…, ํ—ˆ์œ„ ์ •๋ณด ๋“ฑ์„ ํฌ๊ด„ํ•˜๋Š” ์ ๋Œ€์  ํ…Œ์ŠคํŠธ ์Šค์œ„ํŠธ๋ฅผ ์ž๋™์œผ๋กœ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.

Giskard Scan์„ ์‚ฌ์šฉํ•˜์—ฌ:

  • ์—์ด์ „ํŠธ ๋ ˆ๋“œํŒ€ ์ˆ˜ํ–‰ โ€” OWASP LLM Top-10 ์œ„ํ˜‘ ๋ฒ”์ฃผ ์ „๋ฐ˜์— ๊ฑธ์ณ ์ ๋Œ€์  ์ž…๋ ฅ์„ ์ž๋™์œผ๋กœ ์ƒ์„ฑ
  • ํ”„๋กฌํ”„ํŠธ ์ธ์ ์…˜ ํ”„๋กœ๋ธŒ ์‹คํ–‰ โ€” ๋ฐ”๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ์ธ์ ์…˜ ํŽ˜์ด๋กœ๋“œ์˜ ๋‚ด์žฅ ๋ฐ์ดํ„ฐ์…‹
  • ์‚ฌ์šฉ์ž ์ •์˜ ์ƒ์„ฑ๊ธฐ๋กœ ํ™•์žฅ โ€” ์ž์ฒด ScenarioGenerator ์ธ์Šคํ„ด์Šค๋ฅผ generate_suite์— ์ „๋‹ฌํ•˜๊ฑฐ๋‚˜ vulnerability_suite_generator_registry์— ๋“ฑ๋ก

๋น ๋ฅธ ์‹œ์ž‘

import asyncio
from giskard.scan import vulnerability_scan


async def my_agent(inputs: str) -> str:
    # Replace with your agent / model call
    return f"Echo: {inputs}"


async def main() -> None:
    await vulnerability_scan(
        target=my_agent,
        description="A customer support chatbot for an e-commerce platform.",
        languages=["en"],
    )


asyncio.run(main())

์Šค์บ” ์ƒ์„ฑ๊ธฐ ์—ญ์‹œ LLM ์ œ๊ณต์ž ์ถ”๊ฐ€ ํŒจํ‚ค์ง€์™€ API ํ‚ค๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค(์œ„์˜ Checks ํŒ์ •๊ธฐ์™€ ๋™์ผ).

Giskard v2๋ฅผ ์ฐพ๊ณ  ๊ณ„์‹ ๊ฐ€์š”?

Giskard v2์—๋Š” Scan(์ž๋™ ์ทจ์•ฝ์  ํƒ์ง€)๊ณผ RAGET(RAG ํ‰๊ฐ€ ํ…Œ์ŠคํŠธ ์„ธํŠธ ์ƒ์„ฑ)์ด ํฌํ•จ๋˜์–ด ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค.

LLM ์—์ด์ „ํŠธ์˜ ๊ฒฝ์šฐ, v3์—์„œ๋Š” ๋‘ ๊ธฐ๋Šฅ ๋ชจ๋‘ giskard-scan์œผ๋กœ ๋Œ€์ฒด๋˜์—ˆ์Šต๋‹ˆ๋‹ค. v2์˜ LLM ์Šค์บ” ๋Œ€์‹  vulnerability_scan์„ ์‚ฌ์šฉํ•˜๊ณ , RAGET ๋Œ€์‹  quality_scan(KnowledgeBase์™€ ํ•จ๊ป˜)์„ ์‚ฌ์šฉํ•˜์„ธ์š”.

v3๋Š” ML ๋ชจ๋ธ๊ณผ๋„ ํ•จ๊ป˜ ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค. ๋ชจ๋ธ์„ ํƒ€๊นƒ์œผ๋กœ ๊ฐ์‹ธ์„œ giskard-checks ๋˜๋Š” giskard-scan์œผ๋กœ ํ‰๊ฐ€ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค. ์•„๋ž˜ ์˜ˆ์ œ์—์„œ ๋‹ค๋ฃจ๋Š” ๋‚ด์šฉ์€ v2 ์ „์šฉ ์ž๋™ ํ…Œ์ด๋ธ”ํ˜• ์Šค์บ” โ€” giskard.Model + giskard.Dataset์„ ๊ฒ€์‚ฌํ•˜์—ฌ ์„ฑ๋Šฅ, ํŽธํ–ฅ, ๊ฒฌ๊ณ ์„ฑ ๋ฌธ์ œ๋ฅผ ์ž๋™ ํƒ์ง€ํ•˜๋Š” ํƒ์ง€๊ธฐ ๋ชจ์Œ โ€” ๊ณผ giskard.testing ML ํ…Œ์ŠคํŠธ ๋ชจ์Œ, ๊ทธ๋ฆฌ๊ณ  Giskard Hub์ž…๋‹ˆ๋‹ค. ์ด๋“ค์€ v3์— ๊ณ„ํš๋˜์–ด ์žˆ์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

pip install "giskard[llm]>2,<3"

Scan โ€” ์„ฑ๋Šฅ, ํŽธํ–ฅ ๋ฐ ๋ณด์•ˆ ๋ฌธ์ œ๋ฅผ ์ž๋™์œผ๋กœ ํƒ์ง€

๋ชจ๋ธ์„ ๊ฐ์‹ธ๊ณ  ์Šค์บ”์„ ์‹คํ–‰ํ•˜์„ธ์š”:

import giskard
import pandas as pd

์นดํ…Œ๊ณ ๋ฆฌ