Max Ploner
Photo: SCIoI
Max Ploner studied computer science at Humboldt University of Berlin and Technical University of Berlin focusing on machine learning. He now works as a PhD student under the supervision of Professor Alan Akbik at HU Berlin. His research interests are concentrated on topological changes during the training of artificial neural networks (ANN). In his master thesis, Max studied growth as a means for reducing the pre-training time of transformer networks. At SCIoI, Max is working on Project 45A (“Modeling Neurogenesis for Continuous Learning”) and researches how ANN growth can improve the continual learning capabilities of these models.
Projects
Max Ploner is member of:
Publications +
6984777
Ploner
1
apa
50
date
desc
year
19984
https://www.scienceofintelligence.de/wp-content/plugins/zotpress/
%7B%22status%22%3A%22success%22%2C%22updateneeded%22%3Afalse%2C%22instance%22%3Afalse%2C%22meta%22%3A%7B%22request_last%22%3A0%2C%22request_next%22%3A0%2C%22used_cache%22%3Atrue%7D%2C%22data%22%3A%5B%7B%22key%22%3A%22FRJM6NEG%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Pohl%20et%20al.%22%2C%22parsedDate%22%3A%222025-08%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BPohl%2C%20S.%2C%20Ploner%2C%20M.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282025%29.%20Towards%20a%20Principled%20Evaluation%20of%20Knowledge%20Editors.%20In%20R.%20Jia%2C%20E.%20Wallace%2C%20Y.%20Huang%2C%20T.%20Pimentel%2C%20P.%20Maini%2C%20V.%20Dankers%2C%20J.%20Wei%2C%20%26amp%3B%20P.%20Lesci%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BProceedings%20of%20the%20First%20Workshop%20on%20Large%20Language%20Model%20Memorization%20%28L2M2%29%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%2047%26%23x2013%3B60%29.%20Association%20for%20Computational%20Linguistics.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.l2m2-1.4%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.l2m2-1.4%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22Towards%20a%20Principled%20Evaluation%20of%20Knowledge%20Editors%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Sebastian%22%2C%22lastName%22%3A%22Pohl%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Robin%22%2C%22lastName%22%3A%22Jia%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Eric%22%2C%22lastName%22%3A%22Wallace%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Yangsibo%22%2C%22lastName%22%3A%22Huang%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Tiago%22%2C%22lastName%22%3A%22Pimentel%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Pratyush%22%2C%22lastName%22%3A%22Maini%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Verna%22%2C%22lastName%22%3A%22Dankers%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Johnny%22%2C%22lastName%22%3A%22Wei%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Pietro%22%2C%22lastName%22%3A%22Lesci%22%7D%5D%2C%22abstractNote%22%3A%22Model%20editing%20has%20been%20gaining%20increasing%20attention%20over%20the%20past%20few%20years.%20For%20Knowledge%20Editing%20in%20particular%2C%20more%20challenging%20evaluation%20datasets%20have%20recently%20been%20released.%20These%20datasets%20use%20different%20methodologies%20to%20score%20the%20success%20of%20editors.%20Yet%2C%20it%20remains%20under-explored%20how%20robust%20these%20methodologies%20are%20and%20whether%20they%20unfairly%20favor%20some%20editors.%20Moreover%2C%20the%20disruptive%20impact%20of%20these%20editors%20on%20overall%20model%20capabilities%20remains%20a%20constant%20blind%20spot.We%20address%20both%20of%20these%20problems%20and%20show%20that%20choosing%20different%20metrics%20and%20evaluation%20methodologies%20as%20well%20as%20different%20edit%20batch%20sizes%20can%20lead%20to%20a%20different%20ranking%20of%20knowledge%20editors.%20Crucially%20we%20demonstrate%20this%20effect%20also%20on%20general%20language%20understanding%20tasks%20evaluated%20alongside%20the%20knowledge%20editing%20tasks.%20Further%20we%20include%20a%20manual%20assessment%20of%20the%20string%20matching%20based%20evaluation%20method%20for%20knowledge%20editing%20that%20is%20favored%20by%20recently%20released%20datasets%2C%20revealing%20a%20tendency%20to%20produce%20false%20positive%20matches.%22%2C%22proceedingsTitle%22%3A%22Proceedings%20of%20the%20First%20Workshop%20on%20Large%20Language%20Model%20Memorization%20%28L2M2%29%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222025-08%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2025.l2m2-1.4%22%2C%22ISBN%22%3A%22979-8-89176-278-7%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2025.l2m2-1.4%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-29T10%3A40%3A09Z%22%7D%7D%2C%7B%22key%22%3A%22P2MBVNQV%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Christoph%20et%20al.%22%2C%22parsedDate%22%3A%222025-08%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BChristoph%2C%20D.%2C%20Ploner%2C%20M.%2C%20Haller%2C%20P.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282025%29.%20From%20Data%20to%20Knowledge%3A%20Evaluating%20How%20Efficiently%20Language%20Models%20Learn%20Facts.%20In%20R.%20Jia%2C%20E.%20Wallace%2C%20Y.%20Huang%2C%20T.%20Pimentel%2C%20P.%20Maini%2C%20V.%20Dankers%2C%20J.%20Wei%2C%20%26amp%3B%20P.%20Lesci%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BProceedings%20of%20the%20First%20Workshop%20on%20Large%20Language%20Model%20Memorization%20%28L2M2%29%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%2029%26%23x2013%3B46%29.%20Association%20for%20Computational%20Linguistics.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.l2m2-1.3%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.l2m2-1.3%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22From%20Data%20to%20Knowledge%3A%20Evaluating%20How%20Efficiently%20Language%20Models%20Learn%20Facts%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Daniel%22%2C%22lastName%22%3A%22Christoph%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Patrick%22%2C%22lastName%22%3A%22Haller%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Robin%22%2C%22lastName%22%3A%22Jia%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Eric%22%2C%22lastName%22%3A%22Wallace%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Yangsibo%22%2C%22lastName%22%3A%22Huang%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Tiago%22%2C%22lastName%22%3A%22Pimentel%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Pratyush%22%2C%22lastName%22%3A%22Maini%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Verna%22%2C%22lastName%22%3A%22Dankers%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Johnny%22%2C%22lastName%22%3A%22Wei%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Pietro%22%2C%22lastName%22%3A%22Lesci%22%7D%5D%2C%22abstractNote%22%3A%22Sample%20efficiency%20is%20a%20crucial%20property%20of%20language%20models%20with%20practical%20implications%20for%20training%20efficiency.%20In%20real-world%20text%2C%20information%20follows%20a%20long-tailed%20distribution.%20Yet%2C%20we%20expect%20models%20to%20learn%20and%20recall%20frequent%20and%20infrequent%20facts.%20Sample%20efficient%20models%20are%20better%20equipped%20to%20handle%20this%20challenge%20of%20learning%20and%20retaining%20rare%20information%20without%20requiring%20excessive%20exposure.%20This%20study%20analyzes%20multiple%20models%20of%20varying%20architectures%20and%20sizes%2C%20all%20trained%20on%20the%20same%20pre-training%20data.%20By%20annotating%20relational%20facts%20with%20their%20frequencies%20in%20the%20training%20corpus%2C%20we%20examine%20how%20model%20performance%20varies%20with%20fact%20frequency.%20Our%20findings%20show%20that%20most%20models%20perform%20similarly%20on%20high-frequency%20facts%20but%20differ%20notably%20on%20low-frequency%20facts.%20This%20analysis%20provides%20new%20insights%20into%20the%20relationship%20between%20model%20architecture%2C%20size%2C%20and%20factual%20learning%20efficiency.%22%2C%22proceedingsTitle%22%3A%22Proceedings%20of%20the%20First%20Workshop%20on%20Large%20Language%20Model%20Memorization%20%28L2M2%29%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222025-08%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2025.l2m2-1.3%22%2C%22ISBN%22%3A%22979-8-89176-278-7%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2025.l2m2-1.3%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A05%3A34Z%22%7D%7D%2C%7B%22key%22%3A%22GC869RF2%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Garbas%20et%20al.%22%2C%22parsedDate%22%3A%222025-04%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BGarbas%2C%20L.%2C%20Ploner%2C%20M.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282025%29.%20TransformerRanker%3A%20A%20Tool%20for%20Efficiently%20Finding%20the%20Best-Suited%20Language%20Models%20for%20Downstream%20Classification%20Tasks.%20In%20N.%20Dziri%2C%20S.%20%28Xiang%29%20Ren%2C%20%26amp%3B%20S.%20Diao%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BProceedings%20of%20the%202025%20Conference%20of%20the%20Nations%20of%20the%20Americas%20Chapter%20of%20the%20Association%20for%20Computational%20Linguistics%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%20295%26%23x2013%3B302%29.%20Association%20for%20Computational%20Linguistics.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.naacl-demo.25%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.naacl-demo.25%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22TransformerRanker%3A%20A%20Tool%20for%20Efficiently%20Finding%20the%20Best-Suited%20Language%20Models%20for%20Downstream%20Classification%20Tasks%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Lukas%22%2C%22lastName%22%3A%22Garbas%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Nouha%22%2C%22lastName%22%3A%22Dziri%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Sean%20%28Xiang%29%22%2C%22lastName%22%3A%22Ren%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Shizhe%22%2C%22lastName%22%3A%22Diao%22%7D%5D%2C%22abstractNote%22%3A%22Classification%20tasks%20in%20NLP%20are%20typically%20addressed%20by%20selecting%20a%20pre-trained%20language%20model%20%28PLM%29%20from%20a%20model%20hub%2C%20and%20fine-tuning%20it%20for%20the%20task%20at%20hand.%20However%2C%20given%20the%20very%20large%20number%20of%20PLMs%20that%20are%20currently%20available%2C%20a%20practical%20challenge%20is%20to%20determine%20which%20of%20them%20will%20perform%20best%20for%20a%20specific%20downstream%20task.%20With%20this%20paper%2C%20we%20introduce%20TransformerRanker%2C%20a%20lightweight%20library%20that%20efficiently%20ranks%20PLMs%20for%20classification%20tasks%20without%20the%20need%20for%20computationally%20costly%20fine-tuning.%20Our%20library%20implements%20current%20approaches%20for%20transferability%20estimation%20%28LogME%2C%20H-Score%2C%20kNN%29%2C%20in%20combination%20with%20layer%20aggregation%20options%2C%20which%20we%20empirically%20showed%20to%20yield%20state-of-the-art%20rankings%20of%20PLMs%20%28Garbas%20et%20al.%2C%202024%29.%20We%20designed%20the%20interface%20to%20be%20lightweight%20and%20easy%20to%20use%2C%20allowing%20users%20to%20directly%20connect%20to%20the%20HuggingFace%20Transformers%20and%20Dataset%20libraries.%20Users%20need%20only%20select%20a%20downstream%20classification%20task%20and%20a%20list%20of%20PLMs%20to%20create%20a%20ranking%20of%20likely%20best-suited%20PLMs%20for%20their%20task.%20We%20make%20TransformerRanker%20available%20as%20a%20pip-installable%20open-source%20library.%22%2C%22proceedingsTitle%22%3A%22Proceedings%20of%20the%202025%20Conference%20of%20the%20Nations%20of%20the%20Americas%20Chapter%20of%20the%20Association%20for%20Computational%20Linguistics%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222025-04%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2025.naacl-demo.25%22%2C%22ISBN%22%3A%22979-8-89176-191-9%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2025.naacl-demo.25%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-29T10%3A40%3A09Z%22%7D%7D%2C%7B%22key%22%3A%2245KSNRRN%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Ploner%20et%20al.%22%2C%22parsedDate%22%3A%222025-04%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BPloner%2C%20M.%2C%20Wiland%2C%20J.%2C%20Pohl%2C%20S.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282025%29.%20LM-Pub-Quiz%3A%20A%20Comprehensive%20Framework%20for%20Zero-Shot%20Evaluation%20of%20Relational%20Knowledge%20in%20Language%20Models.%20In%20N.%20Dziri%2C%20S.%20%28Xiang%29%20Ren%2C%20%26amp%3B%20S.%20Diao%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BProceedings%20of%20the%202025%20Conference%20of%20the%20Nations%20of%20the%20Americas%20Chapter%20of%20the%20Association%20for%20Computational%20Linguistics%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%2029%26%23x2013%3B39%29.%20Association%20for%20Computational%20Linguistics.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.naacl-demo.4%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2025.naacl-demo.4%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22LM-Pub-Quiz%3A%20A%20Comprehensive%20Framework%20for%20Zero-Shot%20Evaluation%20of%20Relational%20Knowledge%20in%20Language%20Models%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Jacek%22%2C%22lastName%22%3A%22Wiland%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Sebastian%22%2C%22lastName%22%3A%22Pohl%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Nouha%22%2C%22lastName%22%3A%22Dziri%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Sean%20%28Xiang%29%22%2C%22lastName%22%3A%22Ren%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Shizhe%22%2C%22lastName%22%3A%22Diao%22%7D%5D%2C%22abstractNote%22%3A%22Knowledge%20probing%20evaluates%20to%20which%20extent%20a%20language%20model%20%28LM%29%20has%20acquired%20relational%20knowledge%20during%20its%20pre-training%20phase.%20It%20provides%20a%20cost-effective%20means%20of%20comparing%20LMs%20of%20different%20sizes%20and%20training%20setups%20and%20is%20useful%20for%20monitoring%20knowledge%20gained%20or%20lost%20during%20continual%20learning%20%28CL%29.%20In%20prior%20work%2C%20we%20presented%20an%20improved%20knowledge%20probe%20called%20BEAR%20%28Wiland%20et%20al.%2C%202024%29%2C%20which%20enables%20the%20comparison%20of%20LMs%20trained%20with%20different%20pre-training%20objectives%20%28causal%20and%20masked%20LMs%29%20and%20addresses%20issues%20of%20skewed%20distributions%20in%20previous%20probes%20to%20deliver%20a%20more%20unbiased%20reading%20of%20LM%20knowledge.%20With%20this%20paper%2C%20we%20present%20LM-Pub-Quiz%2C%20a%20Python%20framework%20and%20leaderboard%20built%20around%20the%20BEAR%20probing%20mechanism%20that%20enables%20researchers%20and%20practitioners%20to%20apply%20it%20in%20their%20work.%20It%20provides%20options%20for%20standalone%20evaluation%20and%20direct%20integration%20into%20the%20widely-used%20training%20pipeline%20of%20the%20Hugging%20Face%20transformers%20library.%20Further%2C%20it%20provides%20a%20fine-grained%20analysis%20of%20different%20knowledge%20types%20to%20assist%20users%20in%20better%20understanding%20the%20knowledge%20in%20each%20evaluated%20LM.%20We%20publicly%20release%20LM-Pub-Quiz%20as%20an%20open-source%20project.https%3A%5C%2F%5C%2Flm-pub-quiz.github.io%5C%2F%22%2C%22proceedingsTitle%22%3A%22Proceedings%20of%20the%202025%20Conference%20of%20the%20Nations%20of%20the%20Americas%20Chapter%20of%20the%20Association%20for%20Computational%20Linguistics%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222025-04%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2025.naacl-demo.4%22%2C%22ISBN%22%3A%22979-8-89176-191-9%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2025.naacl-demo.4%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-29T10%3A40%3A07Z%22%7D%7D%2C%7B%22key%22%3A%22C2MIIUUN%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Garbaciauskas%20et%20al.%22%2C%22parsedDate%22%3A%222024-08%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BGarbaciauskas%2C%20L.%2C%20Ploner%2C%20M.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282024%29.%20Choose%20Your%20Transformer%3A%20Improved%20Transferability%20Estimation%20of%20Transformer%20Models%20on%20Classification%20Tasks.%20In%20L.-W.%20Ku%2C%20A.%20Martins%2C%20%26amp%3B%20V.%20Srikumar%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BFindings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20ACL%202024%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%2012752%26%23x2013%3B12768%29.%20Association%20for%20Computational%20Linguistics.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-acl.757%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-acl.757%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22Choose%20Your%20Transformer%3A%20Improved%20Transferability%20Estimation%20of%20Transformer%20Models%20on%20Classification%20Tasks%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Lukas%22%2C%22lastName%22%3A%22Garbaciauskas%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Lun-Wei%22%2C%22lastName%22%3A%22Ku%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Andre%22%2C%22lastName%22%3A%22Martins%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Vivek%22%2C%22lastName%22%3A%22Srikumar%22%7D%5D%2C%22abstractNote%22%3A%22There%20currently%20exists%20a%20multitude%20of%20pre-trained%20transformer%20language%20models%20%28LMs%29%20that%20are%20readily%20available.%20From%20a%20practical%20perspective%2C%20this%20raises%20the%20question%20of%20which%20pre-trained%20LM%20will%20perform%20best%20if%20fine-tuned%20for%20a%20specific%20downstream%20NLP%20task.%20However%2C%20exhaustively%20fine-tuning%20all%20available%20LMs%20to%20determine%20the%20best-fitting%20model%20is%20computationally%20infeasible.%20To%20address%20this%20problem%2C%20we%20present%20an%20approach%20that%20inexpensively%20estimates%20a%20ranking%20of%20the%20expected%20performance%20of%20a%20given%20set%20of%20candidate%20LMs%20for%20a%20given%20task.%20Following%20a%20layer-wise%20representation%20analysis%2C%20we%20extend%20existing%20approaches%20such%20as%20H-score%20and%20LogME%20by%20aggregating%20representations%20across%20all%20layers%20of%20the%20transformer%20model.%20We%20present%20an%20extensive%20analysis%20of%2020%20transformer%20LMs%2C%206%20downstream%20NLP%20tasks%2C%20and%20various%20estimators%20%28linear%20probing%2C%20kNN%2C%20H-score%2C%20and%20LogME%29.%20Our%20evaluation%20finds%20that%20averaging%20the%20layer%20representations%20significantly%20improves%20the%20Pearson%20correlation%20coefficient%20between%20the%20true%20model%20ranks%20and%20the%20estimate%2C%20increasing%20from%200.58%20to%200.86%20for%20LogME%20and%20from%200.65%20to%200.88%20for%20H-score.%22%2C%22proceedingsTitle%22%3A%22Findings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20ACL%202024%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222024-08%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2024.findings-acl.757%22%2C%22ISBN%22%3A%22%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2024.findings-acl.757%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A01%3A04Z%22%7D%7D%2C%7B%22key%22%3A%22YVLKMAPQ%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Ploner%20and%20Akbik%22%2C%22parsedDate%22%3A%222024-03%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BPloner%2C%20M.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282024%29.%20Parameter-Efficient%20Fine-Tuning%3A%20Is%20There%20An%20Optimal%20Subset%20of%20Parameters%20to%20Tune%3F%20In%20Y.%20Graham%20%26amp%3B%20M.%20Purver%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BFindings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20EACL%202024%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%201743%26%23x2013%3B1759%29.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-eacl.122%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-eacl.122%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22Parameter-Efficient%20Fine-Tuning%3A%20Is%20There%20An%20Optimal%20Subset%20of%20Parameters%20to%20Tune%3F%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Yvette%22%2C%22lastName%22%3A%22Graham%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Matthew%22%2C%22lastName%22%3A%22Purver%22%7D%5D%2C%22abstractNote%22%3A%22The%20ever-growing%20size%20of%20pretrained%20language%20models%20%28PLM%29%20presents%20a%20significant%20challenge%20for%20efficiently%20fine-tuning%20and%20deploying%20these%20models%20for%20diverse%20sets%20of%20tasks%20within%20memory-constrained%20environments.In%20light%20of%20this%2C%20recent%20research%20has%20illuminated%20the%20possibility%20of%20selectively%20updating%20only%20a%20small%20subset%20of%20a%20model%26%23039%3Bs%20parameters%20during%20the%20fine-tuning%20process.Since%20no%20new%20parameters%20or%20modules%20are%20added%2C%20these%20methods%20retain%20the%20inference%20speed%20of%20the%20original%20model%20and%20come%20at%20no%20additional%20computational%20cost.%20However%2C%20an%20open%20question%20pertains%20to%20which%20subset%20of%20parameters%20should%20best%20be%20tuned%20to%20maximize%20task%20performance%20and%20generalizability.%20To%20investigate%2C%20this%20paper%20presents%20comprehensive%20experiments%20covering%20a%20large%20spectrum%20of%20subset%20selection%20strategies.%20We%20comparatively%20evaluate%20their%20impact%20on%20model%20performance%20as%20well%20as%20the%20resulting%20model%26%23039%3Bs%20capability%20to%20generalize%20to%20different%20tasks.Surprisingly%2C%20we%20find%20that%20the%20gains%20achieved%20in%20performance%20by%20elaborate%20selection%20strategies%20are%2C%20at%20best%2C%20marginal%20when%20compared%20to%20the%20outcomes%20obtained%20by%20tuning%20a%20random%20selection%20of%20parameter%20subsets.%20Our%20experiments%20also%20indicate%20that%20selection-based%20tuning%20impairs%20generalizability%20to%20new%20tasks.%22%2C%22proceedingsTitle%22%3A%22Findings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20EACL%202024%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222024-03%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2024.findings-eacl.122%22%2C%22ISBN%22%3A%22%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2024.findings-eacl.122%5C%2F%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A00%3A59Z%22%7D%7D%2C%7B%22key%22%3A%223AL7FFIA%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Wiland%20et%20al.%22%2C%22parsedDate%22%3A%222024%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BWiland%2C%20J.%2C%20Ploner%2C%20M.%2C%20%26amp%3B%20Akbik%2C%20A.%20%282024%29.%20BEAR%3A%20A%20Unified%20Framework%20for%20Evaluating%20Relational%20Knowledge%20in%20Causal%20and%20Masked%20Language%20Models.%20%26lt%3Bi%26gt%3BFindings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20NAACL%202024%26lt%3B%5C%2Fi%26gt%3B%2C%202393%26%23x2013%3B2411.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-naacl.155%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.18653%5C%2Fv1%5C%2F2024.findings-naacl.155%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22BEAR%3A%20A%20Unified%20Framework%20for%20Evaluating%20Relational%20Knowledge%20in%20Causal%20and%20Masked%20Language%20Models%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Jacek%22%2C%22lastName%22%3A%22Wiland%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Max%22%2C%22lastName%22%3A%22Ploner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Alan%22%2C%22lastName%22%3A%22Akbik%22%7D%5D%2C%22abstractNote%22%3A%22%22%2C%22proceedingsTitle%22%3A%22Findings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20NAACL%202024%22%2C%22conferenceName%22%3A%22Findings%20of%20the%20Association%20for%20Computational%20Linguistics%3A%20NAACL%202024%22%2C%22date%22%3A%222024%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.18653%5C%2Fv1%5C%2F2024.findings-naacl.155%22%2C%22ISBN%22%3A%22%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Faclanthology.org%5C%2F2024.findings-naacl.155%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22en%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T11%3A56%3A07Z%22%7D%7D%5D%7D
Pohl, S., Ploner, M., & Akbik, A. (2025). Towards a Principled Evaluation of Knowledge Editors. In R. Jia, E. Wallace, Y. Huang, T. Pimentel, P. Maini, V. Dankers, J. Wei, & P. Lesci (Eds.), Proceedings of the First Workshop on Large Language Model Memorization (L2M2) (pp. 47–60). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.l2m2-1.4
Christoph, D., Ploner, M., Haller, P., & Akbik, A. (2025). From Data to Knowledge: Evaluating How Efficiently Language Models Learn Facts. In R. Jia, E. Wallace, Y. Huang, T. Pimentel, P. Maini, V. Dankers, J. Wei, & P. Lesci (Eds.), Proceedings of the First Workshop on Large Language Model Memorization (L2M2) (pp. 29–46). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.l2m2-1.3
Garbas, L., Ploner, M., & Akbik, A. (2025). TransformerRanker: A Tool for Efficiently Finding the Best-Suited Language Models for Downstream Classification Tasks. In N. Dziri, S. (Xiang) Ren, & S. Diao (Eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (pp. 295–302). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-demo.25
Ploner, M., Wiland, J., Pohl, S., & Akbik, A. (2025). LM-Pub-Quiz: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models. In N. Dziri, S. (Xiang) Ren, & S. Diao (Eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (pp. 29–39). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-demo.4
Garbaciauskas, L., Ploner, M., & Akbik, A. (2024). Choose Your Transformer: Improved Transferability Estimation of Transformer Models on Classification Tasks. In L.-W. Ku, A. Martins, & V. Srikumar (Eds.), Findings of the Association for Computational Linguistics: ACL 2024 (pp. 12752–12768). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-acl.757
Ploner, M., & Akbik, A. (2024). Parameter-Efficient Fine-Tuning: Is There An Optimal Subset of Parameters to Tune? In Y. Graham & M. Purver (Eds.), Findings of the Association for Computational Linguistics: EACL 2024 (pp. 1743–1759). https://doi.org/10.18653/v1/2024.findings-eacl.122
Wiland, J., Ploner, M., & Akbik, A. (2024). BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models. Findings of the Association for Computational Linguistics: NAACL 2024, 2393–2411. https://doi.org/10.18653/v1/2024.findings-naacl.155
Science Mail – August 2025 - Вместе лучше: у рыб в Мексике обнаружили коллективный интеллект
Vista al Mar – August 2025 - Mejor juntos y muchos
