Skip to content

Instantly share code, notes, and snippets.

@karlseguin
Last active September 30, 2023 06:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save karlseguin/4518694523af63f5edec8048049b95ec to your computer and use it in GitHub Desktop.
Save karlseguin/4518694523af63f5edec8048049b95ec to your computer and use it in GitHub Desktop.
Zig's utf8ValidateSlice vs Go's implementation
const std = @import("std");
const mem = std.mem;
const ITERATIONS = 20_000;
pub fn main() !void {
try benchmark("empty", "", true);
try benchmark("short.ascii", "it's over 9000!!", true);
try benchmark("long.ascii", "Lorem ipsum dolor sit amet, pri legere suscipit repudiare at, cu vix aliquip pertinacia quaerendum, in eam dicat cetero eripuit. Duo odio nonumy viderer at, et sea vidit dolor animal, has persius conceptam id. Vide idque copiosae ut qui, ut lorem atomorum mei, in eos suscipit eleifend. Illud principes no eos, pri dolorem singulis ne, ex pri malorum argumentum scribentur. Et altera regione per. Disputando contentiones no mei, et vel discere meliore constituam. Te quo legere nostrum deleniti, ad eros putent tritani per.Repudiandae mediocritatem ut est, ex sea dicit evertitur. Per tritani detraxit an. Pro no fastidii tractatos, per purto affert accusata no, affert consequat mel ea. Cum ex postulant gubergren, has quot vidit argumentum no.Eu est esse feugait, mea no augue debet constituam. Sea stet augue libris no, affert fierent in sea, mei soluta cetero corrumpit ex. Iriure saperet has ut. Quem mucius maiestatis ea mea. Cum eu partiendo complectitur. Ei quo veri lobortis percipitur, te velit maluisset tincidunt mel. Has ei debet munere menandri.Tempor sententiae eu mel, id mei facete necessitatibus, qui ei vocibus commune. Case minimum an nam. Ut magna solet democritum ius, eu vix iudico eirmod ceteros. Has ea labore voluptua postulant. Reque atqui vocent an sit, usu tritani dissentias ei, ius id aperiri elaboraret disputando. Vis cu audire neglegentur, viderer habemus offendit pro et, sed at forensibus voluptaria philosophia.Pro no hinc cibo euripidis. Eum in eirmod expetendis, cibo aeque has ei, duo at graeco eripuit assentior. Mei omnis partem debitis eu, latine consetetur has ei. Vix primis repudiandae te, qui vide assentior te. Vel amet tollit consequuntur id. Facilisi principes ad mei, congue blandit contentiones eos at, ius justo honestatis eu.Tractatos disputando at eum, pro cu iisque epicurei. Integre aliquam scaevola ei vis, te euismod meliore qui. Ius scripta detracto disputationi cu. Ea vel expetenda dissentiunt, primis iracundia adversarium nam eu, meis ubique efficiantur est ea. Eos nominavi inimicus torquatos in, eum graecis scribentur vituperatoribus ne, amet eloquentiam vel ut.At luptatum accusata signiferumque mea. Cibo omnis ut mea. Sea meis primis commune ad, quot facer disputationi eum eu. Eu probo ridens sit, nam ei cibo fabulas, integre aliquid invidunt no usu.Ea wisi fuisset lobortis eos, eu facer corrumpit sed. Te cetero petentium persecuti duo. Duo te mucius inermis adversarium, mea no modus percipitur. Ad tation appetere cum, quo latine dolorem te. Aliquando definiebas necessitatibus te ius, te est error albucius postulant.Laoreet molestiae gloriatur id per, ut vis oporteat dissentias. Ius admodum abhorreant ad, duo no primis minimum sapientem. Eam ne scaevola apeirian adipiscing, cu mea causae detracto percipitur. Ut movet invenire his, eu mea tota docendi accusata, no singulis lobortis ullamcorper eos.Option virtute adversarium quo ei, sed ex mandamus efficiendi. Ad per omnium dissentias. Sonet vocibus eu sit, ut soleat menandri mei. Nec cu noster tincidunt, vidit legere ex sed. Viderer discere cum ea.Ad vis postea labitur suscipiantur, temporibus contentiones duo at. Cu admodum mentitum eos, viris luptatum interesset pro ad, ea nominati erroribus cum. Vix nonumy ridens ut, ut cum equidem maiorum. Mei ex iudico accusam, mentitum antiopam ut est. Docendi consulatu vis ut, vel ut dicit accumsan, est mundi epicuri intellegam no.Duo odio vivendum contentiones in, cu enim accusata voluptatibus quo. Quando alterum ancillae ne eam, nisl tractatos intellegam ne per. Eu vide percipitur qui, ius ad utamur persecuti. Purto deleniti sed ad, quis urbanitas has ei. Ad vel dolorem tractatos repudiare.Oratio aperiam suavitate nam ei, cu probo aliquando accommodare sit. Pro ei repudiare cotidieque, aperiri nostrum expetendis ei pro. Habeo quodsi salutatus sit no, per decore aliquam cu. Nostrum efficiantur an est. Ei ius dico nibh vidisse.Tantas ubique scripserit te quo, decore labitur insolens eu vim. Ad vix fugit conceptam dissentias, eam posse ipsum nullam te. Quo at recusabo accusamus, nam eu enim feugiat, ius ad postulant disputando comprehensam. Iusto eruditi epicurei at mei, his te phaedrum deseruisse. An his congue veniam, id vix erat fierent, adipisci sensibus id eam. Ei quem sensibus gubergren nec, at per soleat accommodare. Viderer nominavi rationibus his ei, per et etiam scriptorem.Cetero scripserit vim ad. Ea case modus comprehensam cum, viris audire ius cu. Veri albucius ius ex, ei solum expetendis intellegebat nec. Pri ne ancillae consulatu urbanitas. Nisl debet vis ad, ei mea graecis maluisset accusamus. Scripta euismod vivendum est no. Duo ludus pertinacia ea, an nonumy detracto nam.Veniam fuisset dolores ei nam, cum eu soluta essent graecis. Sale dictas est eu. An denique omnesque molestie sea, unum labore deterruisset mea no, dolore persecuti ut est. Et sit dictas maluisset. Ex quod solum euismod his.Cum affert scaevola gloriatur no, modus saperet similique id pri. Mutat ludus in per, per at erat lobortis elaboraret. Vis cetero ancillae ne, his rationibus suscipiantur necessitatibus ne. Eum dicant possit eleifend no. Ne paulo numquam per. An dicta nonumes eos.Mea quidam complectitur et, nihil noster et mel, vim an debet quaestio. Te ius fugit aliquid corpora, vim animal deterruisset an. Cum at nusquam omittantur, et sit assum repudiare, mei molestie sadipscing omittantur an. Sit viris euismod cu, vis tation legere percipit ad. Oratio aperiri laboramus eos no. Ne postea voluptatibus mel.Vim ad viris elitr iudicabit, ea quot debet cotidieque vel. Purto vitae nonumes id has. Eruditi philosophia vel an. Eum at primis impetus ullamcorper, mei quot percipit argumentum at. Tota admodum molestie vis ad, eu est percipit interesset, legimus indoctum pertinacia id his. Gubergren philosophia cu sed, eum ei erant soleat. Ex vix detraxit vituperata, liber persius voluptua qui ne. Ad mel qualisque concludaturque, docendi pertinax ea eos, ei sit iudico democritum interesset. Ut albucius vivendum disputationi sit, senserit mnesarchum usu ut. Facilisis argumentum necessitatibus nec ei, nonumes accusamus vis ea.", true);
try benchmark("long.chinese", "公九改魅娠形美江事児新社欄管。間果策巧馬記告直部月禁脇立。施毎売歌個戒投読忘新暮油郷優算台。克表著域書詳文伝一和訴場変暮。原引放求清張話歩少最族介人土。記装市徳一気支腐縁独元感外希募年新。引査足記准本論恵一仁白収寛木豊反面中注負。艶家仕型編暖時市季試転南年票暮書誉私相外。常愛役井陵安流事事口王域毎。辺違大与討三炒体同霊氷無区貯削元校説。倒美島落際柔人武予家静精改。読困融校展府氏上四賀致線育。坦相新手類宅徹的沿情写業京総置個破情。残位行属官任減医質件認治私回渋員支野真。情未就論価製送告速校権譜。東渡聞裕同読江碁提芸度績誌町。面代康聞難賞一健型理変島馬起化関。職当米注催上拠教文紙決姿利線余現己損称。来貴技百以育赤規禁市上海医読齢指念実変。四光目禁初得観本学欧詳森。用全幌考民会策計門流展事栄日一念稿事作。題視道水技真録新意無車合事物情正同落主針。崎浜除能者選権城止代芸掲治関生。写康治賞終強本静監案評革事上円球。哲引坊無三粉牧素馬生供歳茶手念横伝中結。射通理暑院外社神最合現属雄性名文。情株責哉技北安皇任済話首電社。有済落新前図傷援七比愚断子戦東症容署保。保男成予需徳火配果毎伝伊交二捕農國。要許作土康質傾返会業集本書合金速冷毎異。響読国覧転出掘型追客岸氏拘野退残期。悪食励芸地写牛評向後形投示方。面更揺団患賞造管再実則無能富測聞真持達。覧八豊要日隊野変企入購広触獲掲重現理。増税経安返遺若康惑作点皇調新多針芝三上。題搾一拳張及防意日土経組事広問外変。思集降春郊情指江国週長末育働時久能下葉。暴度投文魅権絶烈向引通売書解見新鍵忠。山質針恐氏壁権海見注来口長高。覧力型要売人暮禁事証護党統査。最場額員発石延女視郎今好際読。要航外球済左障購援供親男弓道分楽。報禁急筋設結意賞計開人俳現向詐。明業意詳張回飯文情意月殺満。少建闘分美打子提爆芸凍表伝。人棄府井犯督真通厚知連代止。上後発拉創無立理銀神死提独。読編十械京協走囲権任院構舎惑付。選田解由知崎先春吉航演化疑輪島交断新経市。見夫長減投関学価特方導的棋見。江食呼頂億情玄領庫購直中購募特。技両事武打境前通更青自秀。学共肢験埼更民理協稿無施。鈴面今藤味景表像町偏証点区王堂多意理密園。月点要紙省覧続出集良南的住。面比見文省就怪喜激施級訴必。求遠育土曜復約集督案督変部一口国至際質。法刊林来分盾易正職康燃股旅。物閣化売業念宏言乗案瞬年。支投村頼社当更民理橿軽億高策。言仕臣察老出新外供減全投機岩微比回越躍億。制女見載毎政族展子比速士車績月検芸面。原高経験開棋明断情態役公後取訃氏月詳入立。追関制西半転芸住堀必園典純環。年企画需九可埼染集融是失。断音原減東請部者盾昇自宮環真改。正録作圧権理左光小国終紀報。速休尻供球人報告計図革問死養。試公面憩首月指止日親真北読季学造茂意応。万地写安意記素夕嶋瀬写界車小。万完率犯切質面症重広部齢信最野定像。技助歩車明通般堀覧確一配退産。昭載井話気愛芸米更形更大点幻索朝組一。気作過宿通暮未台変案島左。常際属計振済給場全売籍姫。願久高市軽院停協歩属関転時想治日補携集。同逮監尾覧告広改不止専珍攻漁葉。結越尚当玲名置連進治請都検報光。行占一場針通増区米入稿希定優軟初。面緒検海茂育年理音令就芸自井。内護政入由山時成方路闘拳。止明金話江産首力超肝曜社断色心月児。問少与門通常五具温以大怪就設。応小色興霊場本王型皇整光信大文。人策長愛昭情工県著陵演婚道共詳集共光。常規出辞条響歩夏転述屋博荘無。隣上覧定日奪端京星暮民敬子加汁崎。作目河応属五点完川話琢医。報東新鎖断際期匠討育去響入道歳。山要済民草同役頑伯日段界体豊田作読四役。止済岩実無問気他金分然郎登以天事賃務復日。表魅見終覧嶋他謙住雨過全潮。四込持月世慢日秀見視暫鈴必三花。充抑後曲言要慎日八出議安。遇当軽食本当楽覧能米曜余。派備堀隠機呆本町育見避本派止示。最旅東界駅合多金投読農要遺朝相体辞品。目季急指情重価論度候法談時欧准全真。変頼英夜性年泳難入害反正戦部。内同積東面事査修検世水能着供掲後館教。合吉見制院需向著期大窮躍自存。替対獲事散名躍下視割南職最長。物幌右属書祝備三上野新史捕質当祥要。発者芸発連阪検目需職能削俳開本佐作問今新。事味官著記右害写約際案図安禁。転消番至葉護工紹更建荒型霊。基質選切集破感写週反級馬欄。方名間止方江歩断変白気限士媛歩諸銀。果年害海副道貿明頭著館直無落送描拉済摩実。可建転験浴刈写会密群強覚乗体少断界継人政。応公亡多株球赤社欧北退献策芭慮泰。伯論浜考女画資掲中部用香並不戦提。懇天総結誤康玲和田保限堀政要。申所軍夢由識演文究記陣家率負一。中返南最円牛野教部元消監洋敏投記横公化。紙提年案仮報余溶高作毎毎南官設万作投新報。強積田鄒供予情長倉上止年回菱映持。空接県廃刻押申成無胎有隠必。芸朴画二投颯日必死覚量申果通武村著望任。稿打書季見任観事分門供木洲形満聞先称。重今猟古展月関第味能声客。断八野使越機読家宿四問丸蔵経携売書。下口業歳能載反覧昨禁競口毎案。止使組属速渡軽環戦観圧惑上学。病客午和伝住環規宇碁館面侵。超掘速修犯三属社点天住覧。環責普昨化企勧他飯定以手族能了。駄全鳥美組死査径部智座地。描聞昨比頼阪時南政変主正。質無周授隠閣京問際要毎注略厚広。断圏付誌伝月却治伸者地気情告。際戒止検戦無細使汐供前選配海景戒驚。犬本戦号壇株目員携報下能米責小原冒終京。属更国省社口応未事意東約出争場病報取映。打内詳注世身子同掲教思政。変朝書防話位在院回題真手年込。合読断古禁的愛危北別芸願野標。掲行翌止違意東果対方上悪真初去参庁。著験伸座横督毎職化上問逃追投能門欠川更。委減磨者法京教提決子画将設覚。掲私足場収同館度走給向氏飛施上橋。不側集会前望海宇木断方寂辺貸。政集政悪敷文解息岡企芸養民入激窓。風室詳泳生立援千想可付気済包。硬保職幣合学上話両反有県害受座神鹿合。速光雪皮償界情港賢音巡応。展第援育組斉意無都塁仕速録目議由月載政府。難総死作元本際客沖禁江価領討惜見拓。点泳死承立見道組心必士級課難。緩題解明必田裁写村国結業誓礼容障方。批要的品目刑治氏受人教条井疑思。層索社周見目台報容鶏加奪画健真実動盛潟他。葉燃競囲税暮世宅苦表報約表社預。庶覧芸典行確車市朝性徴人分月写奪特増治。負女最誕惑就藤月調含表縁反弘長会使般込圏。結鹿況日母教件住部報言来海探消記更。要甲録出形項使発約昭宅読治校横都要思記。牲笑山市声万挙認設作諸再軽円死。演史調出内子出女汗才也球。次副東際止江換降敏戦判担。手紀面出繰傑歩車一局担毎。細暮家利配小上障方軽松罪機底変。決無住県切会治事碁就電毎育総報協分下。園京止実元対般並通越応日工欧君文観財皇我。訴当月努震肩期政長半稿負方星暮線。超状注倒研間同棚止草性大無質疑探江。定茶健歴面本夫先本逃雄聞木。聞法玉秀午中済編式新日考。他果空紀町堀材情命工制識成介決決。駆巻壮情際関実本政管使家毎仕。芸民通自遅己難転属多版弁者文。軒未容健政神静番常正威示謙球。健也竹減況格陳盟物防橋連故取残。本大今入指問持企士恵詳組択拶頭人覧兵機腐。康日広省健係田要設季理験業案挙補企釜。佳度男巧和池詐皇天計名案会知労右。載守盗社稿状倒権壊遠望理渡帯伴英征欧王争。盟王百済農平大投日升中整悪。下気里用回批電抜通域速写更生中載載罪。従取約止尾断名善問意渡更情頼作嶺。巡後会覧疑周専頂優相芸作素代。高専明歯違情充者木条援体航著視。震詳産別載泉近文要選十度奇偲多聞基。時提打転読禁宅医小剤幕屋立負民心目。起石入史調語掲財毎式航日。", true);
try benchmark("short.invalid", "abc\xc0abc", false);
try benchmark("long.invalid", "公九改魅娠形美江事児新社欄管。間果策巧馬記告直部月禁脇立。施毎売歌個戒投読忘新暮油郷優算台。克表著域書詳文伝一和訴場変暮。原引放求清張話歩少最族介人土。記装市徳一気支腐縁独元感外希募年新。引査足記准本論恵一仁白収寛木豊反面中注負。艶家仕型編暖時市季試転南年票暮書誉私相外。常愛役井陵安流事事口王域毎。辺違大与討三炒体同霊氷無区貯削元校説。倒美島落際柔人武予家静精改。読困融校展府氏上四賀致線育。坦相新手類宅徹的沿情写業京総置個破情。残位行属官任減医質件認治私回渋員支野真。情未就論価製送告速校権譜。東渡聞裕同読江碁提芸度績誌町。面代康聞難賞一健型理変島馬起化関。職当米注催上拠教文紙決姿利線余現己損称。来貴技百以育赤規禁市上海医読齢指念実変。四光目禁初得観本学欧詳森。用全幌考民会策計門流展事栄日一念稿事作。題視道水技真録新意無車合事物情正同落主針。崎浜除能者選権城止代芸掲治関生。写康治賞終強本静監案評革事上円球。哲引坊無三粉牧素馬生供歳茶手念横伝中結。射通理暑院外社神最合現属雄性名文。情株責哉技北安皇任済話首電社。有済落新前図傷援七比愚断子戦東症容署保。保男成予需徳火配果毎伝伊交二捕農國。要許作土康質傾返会業集本書合金速冷毎異。響読国覧転出掘型追客岸氏拘野退残期。悪食励芸地写牛評向後形投示方。面更揺団患賞造管再実則無能富測聞真持達。覧八豊要日隊野変企入購広触獲掲重現理。増税経安返遺若康惑作点皇調新多針芝三上。題搾一拳張及防意日土経組事広問外変。思集降春郊情指江国週長末育働時久能下葉。暴度投文魅権絶烈向引通売書解見新鍵忠。山質針恐氏壁権海見注来口長高。覧力型要売人暮禁事証護党統査。最場額員発石延女視郎今好際読。要航外球済左障購援供親男弓道分楽。報禁急筋設結意賞計開人俳現向詐。明業意詳張回飯文情意月殺満。少建闘分美打子提爆芸凍表伝。人棄府井犯督真通厚知連代止。上後発拉創無立理銀神死提独。読編十械京協走囲権任院構舎惑付。選田解由知崎先春吉航演化疑輪島交断新経市。見夫長減投関学価特方導的棋見。江食呼頂億情玄領庫購直中購募特。技両事武打境前通更青自秀。学共肢験埼更民理協稿無施。鈴面今藤味景表像町偏証点区王堂多意理密園。月点要紙省覧続出集良南的住。面比見文省就怪喜激施級訴必。求遠育土曜復約集督案督変部一口国至際質。法刊林来分盾易正職康燃股旅。物閣化売業念宏言乗案瞬年。支投村頼社当更民理橿軽億高策。言仕臣察老出新外供減全投機岩微比回越躍億。制女見載毎政族展子比速士車績月検芸面。原高経験開棋明断情態役公後取訃氏月詳入立。追関制西半転芸住堀必園典純環。年企画需九可埼染集融是失。断音原減東請部者盾昇自宮環真改。正録作圧権理左光小国終紀報。速休尻供球人報告計図革問死養。試公面憩首月指止日親真北読季学造茂意応。万地写安意記素夕嶋瀬写界車小。万完率犯切質面症重広部齢信最野定像。技助歩車明通般堀覧確一配退産。昭載井話気愛芸米更形更大点幻索朝組一。気作過宿通暮未台変案島左。常際属計振済給場全売籍姫。願久高市軽院停協歩属関転時想治日補携集。同逮監尾覧告広改不止専珍攻漁葉。結越尚当玲名置連進治請都検報光。行占一場針通増区米入稿希定優軟初。面緒検海茂育年理音令就芸自井。内護政入由山時成方路闘拳。止明金話江産首力超肝曜社断色心月児。問少与門通常五具温以大怪就設。応小色興霊場本王型皇整光信大文。人策長愛昭情工県著陵演婚道共詳集共光。常規出辞条響歩夏転述屋博荘無。隣上覧定日奪端京星暮民敬子加汁崎。作目河応属五点完川話琢医。報東新鎖断際期匠討育去響入道歳。山要済民草同役頑伯日段界体豊田作読四役。止済岩実無問気他金分然郎登以天事賃務復日。表魅見終覧嶋他謙住雨過全潮。四込持月世慢日秀見視暫鈴必三花。充抑後曲言要慎日八出議安。遇当軽食本当楽覧能米曜余。派備堀隠機呆本町育見避本派止示。最旅東界駅合多金投読農要遺朝相体辞品。目季急指情重価論度候法談時欧准全真。変頼英夜性年泳難入害反正戦部。内同積東面事査修検世水能着供掲後館教。合吉見制院需向著期大窮躍自存。替対獲事散名躍下視割南職最長。物幌右属書祝備三上野新史捕質当祥要。発者芸発連阪検目需職能削俳開本佐作問今新。事味官著記右害写約際案図安禁。転消番至葉護工紹更建荒型霊。基質選切集破感写週反級馬欄。方名間止方江歩断変白気限士媛歩諸銀。果年害海副道貿明頭著館直無落送描拉済摩実。可建転験浴刈写会密群強覚乗体少断界継人政。応公亡多株球赤社欧北退献策芭慮泰。伯論浜考女画資掲中部用香並不戦提。懇天総結誤康玲和田保限堀政要。申所軍夢由識演文究記陣家率負一。中返南最円牛野教部元消監洋敏投記横公化。紙提年案仮報余溶高作毎毎南官設万作投新報。強積田鄒供予情長倉上止年回菱映持。空接県廃刻押申成無胎有隠必。芸朴画二投颯日必死覚量申果通武村著望任。稿打書季見任観事分門供木洲形満聞先称。重今猟古展月関第味能声客。断八野使越機読家宿四問丸蔵経携売書。下口業歳能載反覧昨禁競口毎案。止使組属速渡軽環戦観圧惑上学。病客午和伝住環規宇碁館面侵。超掘速修犯三属社点天住覧。環責普昨化企勧他飯定以手族能了。駄全鳥美組死査径部智座地。描聞昨比頼阪時南政変主正。質無周授隠閣京問際要毎注略厚広。断圏付誌伝月却治伸者地気情告。際戒止検戦無細使汐供前選配海景戒驚。犬本戦号壇株目員携報下能米責小原冒終京。属更国省社口応未事意東約出争場病報取映。打内詳注世身子同掲教思政。変朝書防話位在院回題真手年込。合読断古禁的愛危北別芸願野標。掲行翌止違意東果対方上悪真初去参庁。著験伸座横督毎職化上問逃追投能門欠川更。委減磨者法京教提決子画将設覚。掲私足場収同館度走給向氏飛施上橋。不側集会前望海宇木断方寂辺貸。政集政悪敷文解息岡企芸養民入激窓。風室詳泳生立援千想可付気済包。硬保職幣合学上話両反有県害受座神鹿合。速光雪皮償界情港賢音巡応。展第援育組斉意無都塁仕速録目議由月載政府。難総死作元本際客沖禁江価領討惜見拓。点泳死承立見道組心必士級課難。緩題解明必田裁写村国結業誓礼容障方。批要的品目刑治氏受人教条井疑思。層索社周見目台報容鶏加奪画健真実動盛潟他。葉燃競囲税暮世宅苦表報約表社預。庶覧芸典行確車市朝性徴人分月写奪特増治。負女最誕惑就藤月調含表縁反弘長会使般込圏。結鹿況日母教件住部報言来海探消記更。要甲録出形項使発約昭宅読治校横都要思記。牲笑山市声万挙認設作諸再軽円死。演史調出内子出女汗才也球。次副東際止江換降敏戦判担。手紀面出繰傑歩車一局担毎。細暮家利配小上障方軽松罪機底変。決無住県切会治事碁就電毎育総報協分下。園京止実元対般並通越応日工欧君文観財皇我。訴当月努震肩期政長半稿負方星暮線。超状注倒研間同棚止草性大無質疑探江。定茶健歴面本夫先本逃雄聞木。聞法玉秀午中済編式新日考。他果空紀町堀材情命工制識成介決決。駆巻壮情際関実本政管使家毎仕。芸民通自遅己難転属多版弁者文。軒未容健政神静番常正威示謙球。健也竹減況格陳盟物防橋連故取残。本大今入指問持企士恵詳組択拶頭人覧兵機腐。康日広省健係田要設季理験業案挙補企釜。佳度男巧和池詐皇天計名案会知労右。載守盗社稿状倒権壊遠望理渡帯伴英征欧王争。盟王百済農平大投日升中整悪。下気里用回批電抜通域速写更生中載載罪。従取約止尾断名善問意渡更情頼作嶺。巡後会覧疑周専頂優相芸作素代。高専明歯違情充者木条援体航著視。震詳産別載泉近文要選十度奇偲多聞基。時提打転読禁宅医小剤幕屋立負民心目。\xfb\xbf\xbf\xbf\xbf起石入史調語掲財毎式航日", false);
}
fn benchmark(name: []const u8, input: []const u8, expected: bool) !void {
var timer = try std.time.Timer.start();
for (0..ITERATIONS) |_| {
if (std.unicode.utf8ValidateSlice(input) != expected) {
@panic("fail");
}
}
std.debug.print("{s}.std: {d}\n", .{name, timer.lap() / ITERATIONS});
timer.reset();
for (0..ITERATIONS) |_| {
if (utf8ValidateSlice(input) != expected) {
@panic("fail");
}
}
std.debug.print("{s}.go: {d}\n\n", .{name, timer.lap() / ITERATIONS});
}
pub fn utf8ValidateSlice(input: []const u8) bool {
var p = input;
// Fast path. Check for and skip 8 bytes of ASCII characters per iteration.
while (p.len >= 8) {
const first32 = @as(u32, p[0]) | @as(u32, p[1]) << 8 | @as(u32, p[2]) << 16 | @as(u32, p[3]) << 24;
const second32 = @as(u32, p[4]) | @as(u32, p[5]) << 8 | @as(u32, p[6]) << 16 | @as(u32, p[7]) << 24;
if ((first32 | second32) & 0x80808080 != 0) {
// Found a non ASCII byte
break;
}
p = p[8..];
}
const rune_self = 0x80;
const locb = 0b10000000;
const hicb = 0b10111111;
// These names of these constants are chosen to give nice alignment in the
// table below. The first nibble is an index into acceptRanges or F for
// special one-byte cases. The second nibble is the Rune length or the
// Status for the special one-byte case.
const xx = 0xF1; // invalid: size 1
const as = 0xF0; // ASCII: size 1
const s1 = 0x02; // accept 0, size 2
const s2 = 0x13; // accept 1, size 3
const s3 = 0x03; // accept 0, size 3
const s4 = 0x23; // accept 2, size 3
const s5 = 0x34; // accept 3, size 4
const s6 = 0x04; // accept 0, size 4
const s7 = 0x44; // accept 4, size 4
const first = comptime ([_]u8{as} ** 128) ++ ([_]u8{xx} ** 64) ++ [_]u8{
xx, xx, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1,
s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1,
s2, s3, s3, s3, s3, s3, s3, s3, s3, s3, s3, s3, s3, s4, s3, s3,
s5, s6, s6, s6, s7, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx,
};
const n = p.len;
var i: usize = 0;
while (i < n) {
const pi = p[i];
if (pi < rune_self) {
i += 1;
continue;
}
const x = first[pi];
if (x == xx) {
return false; // Illegal starter byte.
}
const size = x & 7;
if (i + size > n) {
return false; // Short or invalid.
}
var accept_lo: u8 = locb;
var accept_hi: u8 = hicb;
switch (x >> 4) {
0 => {},
1 => accept_lo = 0xA0,
2 => accept_hi = 0x9F,
3 => accept_lo = 0x90,
4 => accept_hi = 0x8F,
else => unreachable,
}
const c1 = p[i+1];
if (c1 < accept_lo or accept_hi < c1) {
return false;
}
switch (size) {
2 => i += 2,
3 => {
const c2 = p[i+2];
if (c2 < locb or hicb < c2) {
return false;
}
i += 3;
},
4 => {
const c2 = p[i+2];
if (c2 < locb or hicb < c2) {
return false;
}
const c3 = p[i+3];
if (c3 < locb or hicb < c3) {
return false;
}
i += 4;
},
else => unreachable,
}
}
return true;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment