Jacek Wiland
Photo: SCIoI
Jacek Wiland is a researcher who worked at SCIoI on Project A02, “Neural Representations for Lifelong Learning.” Before joining SCIoI he completed his master’s degree in Statistics at Humboldt Universität in Berlin with specializations in Statistical Inference and Data Science. For his thesis, he examined how factual knowledge encoded in Large Language Models (LLMs) evolves in sequential learning. He is particularly interested in Natural Language Processing (NLP), lifelong learning, multimodal deep learning, and conversational agents.
Projects
Jacek Wiland is member of:
Publications +
6984777
Wiland
1
apa
50
date
desc
year
20054
https://www.scienceofintelligence.de/wp-content/plugins/zotpress/
%7B%22status%22%3A%22success%22%2C%22updateneeded%22%3Afalse%2C%22instance%22%3Afalse%2C%22meta%22%3A%7B%22request_last%22%3A0%2C%22request_next%22%3A0%2C%22used_cache%22%3Atrue%7D%2C%22data%22%3A%5B%7B%22key%22%3A%2245KSNRRN%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Ploner%20et%20al.%22%2C%22parsedDate%22%3A%222025-04%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BPloner%2C%20M.%2C%20Wiland%2C%20J.%2C%20Pohl%2C%20S.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282025%29.%20LM-Pub-Quiz%3A%20A%20Comprehensive%20Framework%20for%20Zero-Shot%20Evaluation%20of%20Relational%20Knowledge%20in%20Language%20Models.%20In%20N.%20Dziri%2C%20S.%20%28Xiang%29%20Ren%2C%20%26amp%3B%20S.%20Diao%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BProceedings%20of%20the%202025%20Conference%20of%20the%20Nations%20of%20the%20Americas%20Chapter%20of%20the%20Association%20for%20Computational%20Linguistics%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%2029%26%23x2013%3B39%29.%20Association%20for%20Computational%20Linguistics.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.naacl-demo.4%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.naacl-demo.4%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22LM-Pub-Quiz%3A%20A%20Comprehensive%20Framework%20for%20Zero-Shot%20Evaluation%20of%20Relational%20Knowledge%20in%20Language%20Models%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Jacek%22%2C%22lastName%22%3A%22Wiland%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Sebastian%22%2C%22lastName%22%3A%22Pohl%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Nouha%22%2C%22lastName%22%3A%22Dziri%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Sean%20%28Xiang%29%22%2C%22lastName%22%3A%22Ren%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Shizhe%22%2C%22lastName%22%3A%22Diao%22%7D%5D%2C%22abstractNote%22%3A%22Knowledge%20probing%20evaluates%20to%20which%20extent%20a%20language%20model%20%28LM%29%20has%20acquired%20relational%20knowledge%20during%20its%20pre-training%20phase.%20It%20provides%20a%20cost-effective%20means%20of%20comparing%20LMs%20of%20different%20sizes%20and%20training%20setups%20and%20is%20useful%20for%20monitoring%20knowledge%20gained%20or%20lost%20during%20continual%20learning%20%28CL%29.%20In%20prior%20work%2C%20we%20presented%20an%20improved%20knowledge%20probe%20called%20BEAR%20%28Wiland%20et%20al.%2C%202024%29%2C%20which%20enables%20the%20comparison%20of%20LMs%20trained%20with%20different%20pre-training%20objectives%20%28causal%20and%20masked%20LMs%29%20and%20addresses%20issues%20of%20skewed%20distributions%20in%20previous%20probes%20to%20deliver%20a%20more%20unbiased%20reading%20of%20LM%20knowledge.%20With%20this%20paper%2C%20we%20present%20LM-Pub-Quiz%2C%20a%20Python%20framework%20and%20leaderboard%20built%20around%20the%20BEAR%20probing%20mechanism%20that%20enables%20researchers%20and%20practitioners%20to%20apply%20it%20in%20their%20work.%20It%20provides%20options%20for%20standalone%20evaluation%20and%20direct%20integration%20into%20the%20widely-used%20training%20pipeline%20of%20the%20Hugging%20Face%20transformers%20library.%20Further%2C%20it%20provides%20a%20fine-grained%20analysis%20of%20different%20knowledge%20types%20to%20assist%20users%20in%20better%20understanding%20the%20knowledge%20in%20each%20evaluated%20LM.%20We%20publicly%20release%20LM-Pub-Quiz%20as%20an%20open-source%20project.https%3A%5C%2F%5C%2Flm-pub-quiz.github.io%5C%2F%22%2C%22proceedingsTitle%22%3A%22Proceedings%20of%20the%202025%20Conference%20of%20the%20Nations%20of%20the%20Americas%20Chapter%20of%20the%20Association%20for%20Computational%20Linguistics%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222025-04%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2025.naacl-demo.4%22%2C%22ISBN%22%3A%22979-8-89176-191-9%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2025.naacl-demo.4%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-29T10%3A40%3A07Z%22%7D%7D%2C%7B%22key%22%3A%223AL7FFIA%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Wiland%20et%20al.%22%2C%22parsedDate%22%3A%222024%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BWiland%2C%20J.%2C%20Ploner%2C%20M.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282024%29.%20BEAR%3A%20A%20Unified%20Framework%20for%20Evaluating%20Relational%20Knowledge%20in%20Causal%20and%20Masked%20Language%20Models.%20%26lt%3Bi%26gt%3BFindings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20NAACL%202024%26lt%3B%5C%2Fi%26gt%3B%2C%202393%26%23x2013%3B2411.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-naacl.155%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-naacl.155%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22BEAR%3A%20A%20Unified%20Framework%20for%20Evaluating%20Relational%20Knowledge%20in%20Causal%20and%20Masked%20Language%20Models%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Jacek%22%2C%22lastName%22%3A%22Wiland%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%5D%2C%22abstractNote%22%3A%22%22%2C%22proceedingsTitle%22%3A%22Findings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20NAACL%202024%22%2C%22conferenceName%22%3A%22Findings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20NAACL%202024%22%2C%22date%22%3A%222024%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2024.findings-naacl.155%22%2C%22ISBN%22%3A%22%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2024.findings-naacl.155%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22en%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T11%3A56%3A07Z%22%7D%7D%5D%7D
Ploner, M., Wiland, J., Pohl, S., & Akbik, A. (2025). LM-Pub-Quiz: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models. In N. Dziri, S. (Xiang) Ren, & S. Diao (Eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (pp. 29–39). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-demo.4
Wiland, J., Ploner, M., & Akbik, A. (2024). BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models. Findings of the Association for Computational Linguistics: NAACL 2024, 2393–2411. https://doi.org/10.18653/v1/2024.findings-naacl.155
