diff --git "a/rag_docs/embeddings_cache.json" "b/rag_docs/embeddings_cache.json"
deleted file mode 100644--- "a/rag_docs/embeddings_cache.json"
+++ /dev/null
@@ -1 +0,0 @@
-[{"content": "# Ablation Panel Guide\n\n## What Is Ablation?\n\nAblation is an experiment where you **remove (disable) specific attention heads** from the model and observe how the output changes. If removing a head causes the prediction to change significantly, that head was important for this particular input. If the prediction barely changes, the head was either redundant or not relevant to this context.\n\nThe term comes from neuroscience, where \"ablation\" means removing part of the brain to study its function.\n\n## How to Use the Ablation Panel\n\nThe ablation panel is found in the **Investigation Panel** at the bottom of the dashboard, under the \"Ablation\" tab.\n\n### Step-by-Step\n\n1. **Run an analysis first**: You need to have clicked \"Analyze\" on a prompt before ablation is available.\n2. **Select a generation for comparison**: Click \"Select for Comparison\" on one of the generated sequences. This gives the ablation experiment a full generation to compare against.\n3. **Choose a head to ablate**: Use the **Layer** and **Head** dropdowns to pick a specific attention head (e.g., Layer 0, Head 3). Click the **+** button to add it.\n4. **Add more heads (optional)**: You can add multiple heads from different layers. Each appears as a chip (e.g., \"L0-H3\") with an \u00d7 button to remove it.\n5. **Run the experiment**: Click \"Run Ablation Experiment.\"\n6. **View results**: The panel shows a side-by-side comparison of the original vs. ablated generation, plus the probability change for the immediate next token.\n\n### Understanding the Results", "source_file": "ablation_panel_guide.md", "chunk_index": 0, "embedding": [0.02053725, 0.03211044, 0.01849644, 0.02764133, -0.01125028, 0.003703809, -0.022875138, 0.025109695, 0.019658927, 0.034022085, 0.027486332, -0.05450767, -0.002268462, 0.011999438, 0.026440095, -0.006671377, 0.052311864, -0.0054862876, 0.02327555, 0.087470606, -0.015861474, 0.011650692, -0.0069167907, 0.03404792, 0.0076659485, -0.018380193, 0.05920928, 0.017411456, 0.03908536, 0.010907993, 0.0077111563, -0.010352583, -0.026130099, -0.0013828737, -0.06349756, -0.037587043, 0.02002059, 0.009351552, -0.028106324, 0.010740078, 0.016081054, -0.036321227, -0.02335305, 0.05853762, -0.016068138, 0.033143762, 0.01813478, -0.0038168286, -0.016752712, 0.034099586, -0.009390302, 0.05605765, -0.022371395, -0.029992135, -0.019142266, 0.04724859, 0.019297265, 0.004359322, 0.026943838, -0.016959377, -0.01420816, 0.042495318, 0.033169597, 0.040170345, -0.017049793, -0.028571319, -0.037277047, 0.03603706, 0.010501122, -0.024748033, 0.041152, -0.015822724, -0.0066326275, 0.023701794, -0.030224632, -0.026000934, 0.0028448612, -0.03133545, -0.03337626, -0.012070478, 0.049160235, 0.028545486, 0.030431297, -0.020679332, -0.0055702445, -0.039266188, -0.057297636, -0.00577368, 0.0039233896, -0.041203666, -0.0128519265, 0.03712205, -0.029217144, 0.038827028, -0.011670066, 0.04492362, -0.0038878694, -0.010313833, -0.0019665386, 0.042701982, 0.030612128, -0.0040913043, -0.019336015, -0.04544028, 0.0041688033, -0.026117181, -0.051304374, 0.0119477715, -0.016093971, 0.024773866, -0.0047597336, -0.03355709, -0.043580305, 0.03174878, -0.008092193, -0.009035098, -0.036398724, -0.007472201, 0.023766378, -0.026414262, -0.067217514, 0.0066326275, 0.030302132, -0.00066035654, -0.012238393, -0.010107169, 0.028364655, -0.030896291, 0.01261943, -0.0034809993, 0.026091348, 0.030431297, -0.03893036, -0.078945704, -0.02468345, 0.034435414, 0.0062451325, 0.030353798, -0.07884237, 0.02138974, 0.015887307, 0.0232368, -0.011431111, 0.00104543, -0.06272257, 0.013768999, 0.051046047, 0.054817665, -0.007975944, 0.05853762, -0.0009477489, 0.03198128, 0.005579932, -0.03851703, -0.023727627, -0.039550353, -0.005221499, -0.006871583, 0.01191548, -0.08767727, 0.037018716, 0.0010422008, -0.0034616245, -0.017824784, -0.042211153, 0.045646943, -0.012684013, 0.018083114, -0.030560462, 0.00984238, -0.03725121, -0.027176335, -0.0025542397, -0.010597996, -0.0034874575, 0.017385622, -0.059415944, 0.0008476459, -0.017514788, -0.04851441, -0.0062451325, 0.007833863, -0.05357768, 0.03944702, -0.028312989, -0.036605388, -0.008886559, -0.017915199, -0.06473754, -0.040945336, 0.03185211, 0.0056735766, 0.010488206, 0.019516846, -0.01954268, -0.0006865932, 0.053112686, 0.010417165, 0.025587605, -0.024179706, 0.023030136, 0.014879819, 0.01480232, 0.018238112, 0.019374764, 0.031542115, -0.015590227, 0.013090882, 0.0072138705, -0.032394607, 0.015977722, -0.059829272, 0.017385622, -0.007833863, -0.035055406, -0.037716206, -0.03670872, 0.048307747, 0.03304043, 0.020408085, 0.01757937, -0.017321039, 0.040015347, 0.06050093, -0.026375512, 0.026233431, 0.039653685, -0.013187756, 0.004207553, -0.034022085, 0.009073848, 0.032885432, -0.001343317, 0.042960312, -0.0041914075, -0.067785844, 0.0004290703, -0.019284349, 0.031774614, -0.044536125, 0.020111006, 0.0072590783, 0.03603706, 0.0010979034, 0.028803816, 0.008034069, 0.0071622045, -0.094755515, 0.016959377, 0.050761882, 0.05450767, 0.02960464, -0.023779294, 0.0012206101, 0.0006361381, 0.022616807, -0.00725262, -0.031206287, -0.026633842, -0.035856232, -0.02083433, -0.0026010622, -0.033247095, 0.008150318, -0.03815537, 0.004485258, 0.003497145, 0.024270121, -0.06752751, -0.027718829, -0.02353388, -0.013588168, 0.013459003, 0.0023637211, -0.03353126, 0.007129913, 0.010455915, -0.004230157, -0.04004118, 0.0014902423, -0.010042586, -0.015486895, 0.014634405, 0.0030176195, -0.026918005, 0.008415106, -0.01946518, -0.005825346, -0.0009735819, 0.006190237, 0.0028367883, -0.037277047, 0.092740536, 0.0055411826, -0.024179706, 0.0007035461, 0.008169692, 0.01480232, -0.027279668, 0.031645447, -0.0044239047, -0.02420554, 0.054146007, -0.02349513, 0.030043801, 0.0005755919, 0.016894795, 0.00973259, -0.014492324, 0.024360538, -0.0031354828, -0.00425599, -0.00018587665, -0.013768999, -0.030147133, -0.008434481, 0.024903031, -0.022733057, -0.017527703, -0.028080491, -0.025432607, -0.07284912, -0.0022813785, 0.011527984, -0.008763852, 0.025329275, -0.009157805, 0.01099195, 0.012625888, -0.005877012, 0.059364278, -0.016223136, -0.0018583628, -0.0044013006, -0.017811866, 0.013936914, -0.040351175, 0.037302878, -0.012219018, 0.01676563, -0.018625606, 0.020821413, -0.04688693, -0.0074463678, -0.023030136, 0.036114562, -0.01054633, -0.010966117, 0.045827776, 0.013129632, -0.023340132, 0.01716604, -0.036941215, -0.037509542, 0.02908798, 0.075587414, -0.008208442, -0.028571319, 0.039472852, -0.007149288, -0.010378416, 0.028803816, 0.002357263, 0.03105129, -0.00043351034, -0.014195244, -0.036837883, 0.023792211, 0.022113064, -0.020227253, 0.023210967, -0.043580305, 0.003778079, -0.039292023, -0.06923249, -0.026065515, 0.0015007369, -0.008408648, -0.025820103, 0.007950111, 0.007556158, -0.026143014, -0.02301722, -0.06463421, 0.05383601, 0.0148927355, -0.016830212, 0.0061450293, -0.06143092, 0.04218532, 0.053009354, -0.012419224, 0.017088542, -0.0018196133, -0.011114656, 0.055230994, -0.02823549, 0.051020212, 0.043916132, 0.011553817, -0.009054474, -0.008847809, -0.022448894, -0.026168847, 0.023391798, 0.0104107065, -0.0103848735, 0.009080307, 0.032472104, 0.024050541, 0.062309243, -0.008931766, 0.03149045, -0.0028884544, 0.02157057, -0.013717333, 0.032213774, 0.055592656, 0.029165478, -0.0077176145, -0.00377485, -0.046680264, 0.003305012, -0.029397976, -0.00083230756, -0.007129913, -0.0021635154, 0.0070653306, 0.04141033, -0.052156866, -0.0074140765, 0.037767872, 0.024166789, -0.036217894, -0.032730434, -0.019891424, 0.005799513, 0.06308423, -0.06447922, 0.015590227, 0.023740545, 0.0019035706, 0.018276861, 0.0010462373, -0.08059902, -0.0039847433, -0.02361138, -0.05357768, -0.004100992, -0.03911119, -0.017605202, -0.0027544457, -0.029914636, 0.07388243, 0.013730249, -0.07496742, 0.059932604, 0.0050051473, 0.062154245, -0.0017840929, 0.037432045, -0.033350427, -0.0032113672, 0.06995582, -0.041875325, 0.036579557, 0.018289778, -0.013446086, -0.007556158, 0.0038588073, 0.002815799, -0.004362551, 0.009144889, -0.011121115, -0.03893036, -0.03141295, 0.016236052, -0.017643953, 0.019374764, -0.057297636, 0.006610024, -0.004381926, 0.028287156, 0.006309715, 0.03918869, 0.06887083, -0.006187008, -0.014117745, 0.006723043, -0.008046986, 0.0018809667, -0.0164298, 0.014104828, -0.013730249, 0.0493669, -0.041126166, -0.035701234, -0.008867184, 0.01998184, 0.005602536, 0.047300257, 0.024877198, 0.009196554, 0.040842004, 0.010662579, -0.020898912, 0.0301213, 0.026840506, -0.008279483, 0.048385244, -0.014673155, -0.010236334, 0.011140489, -0.005805971, -0.014750654, 0.011928396, 0.010468831, -0.021660987, 0.025833018, 0.007782197, -0.013768999, -0.015086483, -0.021790152, 0.023185134, 0.034616247, 0.0111017395, -0.024515536, -0.027357167, -0.05647098, -0.005757534, -0.00069628056, -0.050296888, -0.012503182, 0.042211153, -0.033867087, -0.036398724, 0.018251028, 0.022552226, -0.050012726, -0.0071169967, 0.0232368, 0.012025271, -0.004966398, -0.022229312, 0.016597714, -0.0035294362, 0.041100334, -0.023585547, 0.02250056, 0.008783227, -0.010765911, -0.029191311, 0.018083114, -0.025225943, -0.027253835, 0.021751402, -0.028984647, -0.045130283, 0.0070136646, -0.030508796, 0.0045853606, -0.027486332, -0.016068138, -0.042572815, 0.0038975568, 0.022345562, -0.015667725, 0.0026398117, -0.0164298, -0.008137401, 0.0013957903, -0.010727162, 0.013239422, -0.023120552, 0.026918005, 0.014324409, -0.0016379749, 0.010223418, -0.0153448135, -0.0133556705, -0.019478096, -0.020072255, -0.045311116, 0.03381542, -0.003248502, -0.042391986, -0.0018680502, -0.000648651, -0.042831145, -0.016119804, 0.006923249, -0.030224632, 0.03725121, 0.038982026, -0.0075884494, 0.019245598, 0.024140956, 0.024334705, 0.014014413, 0.008815518, 0.027537998, 0.00016882282, -0.0045789024, 0.048591908, -0.011521527, -0.007355952, -0.035649568, 0.013601084, 0.013536502, 0.0077628223, 0.015964806, -0.012793803, -0.05605765, 0.0016856045, -0.016403968, 0.04448446, 0.013872331, 0.018858103, 0.03851703, -0.014608572, 0.0015620904, -0.00041494286, -0.030302132, 0.0138335815, 0.02272014, 0.02031767, -0.008957599, -0.00517952, 0.003978285, 0.0021974212, -0.0689225, 0.035907898, 0.02264264, 0.018806437, -0.0012189955, 0.03208461, -0.0013352442, -0.016520215, 0.011967146, 0.01339442, -0.0021554425, -0.0012480577, 0.017346872, -0.012593597, -0.011121115, 0.005221499, 0.024063457, -0.02002059, -0.009790714, -0.017941032, 0.009571133, -0.032885432, 0.047429424, 0.013510669, 0.024450952, 0.006768251, 9.3039234e-05, -0.0020133608, 0.0028254865, 0.0054539964, -0.04236615, 0.034926243, 0.003038609, -0.030612128, 0.013562335, 0.009823006, -0.0045240074, 0.0028287156, 0.0037102674, -4.7982816e-05, -0.03725121, 0.009080307, 0.007575533, 0.0046725473, -0.02138974, 0.05672931, 0.011883189, -0.0028400174, -0.023792211, 0.03497791, -0.013484836, 0.024709282, -0.031516284, 0.02841632, -0.0056154523, 0.01247089, 0.01032675, -0.0069426238, -0.003577873, -0.011786315, 0.009319262, -0.0219064, -0.015551478, -0.012057561, -0.020730998, 0.025561772, -0.022319729, 0.03908536, -0.0044884873, -0.08721227, -0.012038187, -0.007375327, 0.017450204, 0.0020036735, -0.015835641, 0.005608994, 0.011779857, 0.049263567, 0.015641892, -0.02994047, -0.032730434, 0.018586857, -0.014711904, 0.020924745, -0.030276299, 0.051304374, -0.026995504, 0.028803816, -0.019568512, -0.047300257, -0.04696443, 0.041461997, -0.047584422, -0.0013513898, 0.049392734, -0.021712653, -0.0044239047, 0.008221358, -0.022745972, -0.0051762913, 0.0153706465, 0.0011406892, -0.0050342097, -0.04184949, 0.008776768, 0.035701234, 0.0022668475, -0.023262633, 0.013090882, -0.022745972, 0.026763007, 0.02242306, 0.034022085, 0.010462373, 0.0031128789, 0.02638843, -0.009196554, -0.0042172405, -0.053371016, -0.016817296, 0.0034035002, 0.010417165, -0.010933826, -0.00012744962, 0.017191874, 0.03037963, -0.009965087, -0.022526393, 0.01813478, 0.009203013, 0.013820665, -0.010036128, 0.02686634, 0.007963028, -0.028338822, -0.029785471, 0.03105129, -0.0013336296, 0.021105576, 0.03446125, -0.006490546, 0.010494664, 0.037328713, 0.018225195, 0.015008984, -0.011482777, 0.04750692, -0.024450952, -0.010688412, 0.026918005, -0.016378134, 0.0047532753, -0.042727813, 0.001761489, -0.0101975845, -0.03397042, -0.031387117, -0.014427741, -0.035313737, -0.019633094, -0.008654062, 0.025161361, 0.012457973, -0.018754771, 0.0124515155, 0.01450524, 0.0071363715, -0.07419243, -0.0014967006, -0.01013946, -0.018121863, -0.036088727, -0.016946461, 0.02815799, 0.0036198518, 0.03319543, 0.0041688033, 0.0010801431, -0.031542115, 0.008847809, -0.012457973, 0.027925493, -0.03327293, 0.010507581, -0.0133815035, 0.03996368, -0.020756831, -0.040067013, 0.01624897, -0.020162672, -0.0014482636, -0.044794455, 0.02756383, -0.03867203, 0.025251776, 0.01058508, 0.03363459, -0.025393859, -0.0038071412, -0.03815537, 0.018586857, -0.02105391, 0.026556343, 0.004937336, 0.0012932655, 3.612586e-05, 0.0014175869, 0.014673155, -0.013471919, -0.0035197488, -0.012625888, 0.023469297, -0.004178491, -0.01728229, 0.013446086, -0.014053162, 0.019839758, 0.0108886175, -0.011682983, 0.008137401, 0.015473979, 0.016701046, -0.036760386, 0.005935136, 0.011586109, 0.003282408, 0.018483525, 0.04458779, 0.05424934, 0.035727065, -0.002150599, -0.019658927, 0.0108886175, 0.010985491, 0.00034067294, -0.05024522, -0.031619616, -0.019865591, 0.03378959, -0.024063457, -0.007782197, 0.019361848, 0.017114375, 0.03859453, 0.028312989, 0.0143631585, -0.014711904, 0.018780604, -0.027202168, -0.04022201, 0.010830494, -0.016403968, 0.030792959, -0.016520215, 0.014595656, 0.013730249, 0.054197673, 0.01535773, -0.011837981, 0.018548107, 0.023185134, -0.037716206, -0.012302975, 0.012580681, 0.012374016, 0.043761134, 0.0013925611, 0.00078629254, -0.007627199, 0.00065833837, -0.015771057, -0.016533133, -0.0021958067, -0.005111709, -0.028261323, -0.0206535, -0.007246162, 0.045207784, 0.045491945, -0.039395355, 0.010300917, -0.015977722, 0.011818606, 0.050451886, 0.008983432, 0.0004783145, -0.010778828, 0.006723043, 0.016610632, -0.0023524193, -0.013265255, 0.032420438, 0.025703853, -0.011340695, -0.021932233, 0.002953037, 0.033763755, 0.016068138, -0.0023927833, -0.03918869, -0.0016460477, -0.0026801757, -0.008428022, -0.01039779, 0.011967146, -0.022113064, 0.00973259, -0.0206535, -0.008279483, -0.0019600803, -0.005999719, -0.06856083, -0.014414825, 0.0014539146, -0.012561305, -0.028028825, 0.027847994, 0.0178377, 0.04339947, -0.016210219, -0.003064442, 0.005835033, 0.033350427, 0.03500374, -0.019865591, -0.041126166, 0.039524518, 0.009093223, 0.0016581569, 0.027331334, 0.0046111937, -0.00035722222, 0.025845936, -0.019710593, 0.0055411826, -0.0019778404, 0.006903874, -0.017450204, 0.0068844995, 0.00836344, 0.025561772, 0.040273678, 0.031387117, -0.024450952, -0.02401179, -0.038827028, 0.009648632, -0.0016274801, 0.025639271, -0.024967613, -0.026181765, -0.031154621, -0.008873642, 0.015125233, -0.012748595, 0.009971545, -0.022913888, 0.016326468, 0.016442716, 0.0026801757, -0.007704698, -0.020149754, 0.0053829555, -0.0219839, 0.0191681, -0.051562704, -0.032678768, -0.032187942, 0.056935973, 0.015667725, -0.013768999, 0.017863533, 0.029837137, 0.009952171, -0.0011067834, -0.024748033, 0.0013965976, -0.03490041, 0.009306345, 0.0035068323, 0.019413514, -0.008537813, 0.02212598, -0.006871583, -0.0128777595, -0.005405559, -0.017631035, -0.04303781, -0.053784344, -0.018147696, 0.016378134, 0.01394983, 0.024618868, 0.025174277, -0.0072074123, 0.004989002, 0.015280231, -0.02357263, -0.009700298, -0.015267314, 0.006645544, -0.029811304, 0.03936952, 0.007950111, -0.0017711764, 0.0133815035, 0.0076401155, 0.044200294, -0.004840462, -0.037948705, -0.009390302, -0.014195244, -0.024812615, 0.030663794, 0.019594345, 0.012509639, -0.04040284, -0.038207036, -0.027822161, 0.0036682887, -0.008395731, 0.012283601, -0.0047791083, 0.03236877, -0.013291088, 0.035442904, 0.0033776672, -0.0438903, 0.031025456, -0.001887425, -0.04492362, 0.01664938, 0.005980344, -0.0356754, -0.022616807, 0.022345562, -0.0031290245, 0.029036313, 0.016029388, -0.014944402, -0.00891885, -0.0046467143, -0.00041857563, 0.0024396058, 0.0018438317, 0.010662579, -0.03371209, 0.041384496, 0.033505425, -0.015525645, 0.01668813, -0.0073365774, -0.019155184, 0.0021990358, 0.012625888, -0.0026656447, -0.019439347, -0.025484273, 0.029837137, 0.019155184, -0.016416883, -0.012651721, 0.02593635, -0.029139645, 0.0073172026, 0.0020230482, -0.037199546, 0.027383, -0.023818044, -0.02005934, -0.014104828, -0.011192155, -0.006574503, 0.003548811, -0.044200294, -0.022655558, 0.028442154, -0.012774428, 0.036321227, 0.017941032, 0.022435976, -0.015254398, -0.029811304, 0.0075367833, -0.041797824, -0.01365275, -0.033143762, -0.016713964, 0.036088727, -0.00943551, 0.008079276, -0.02079558, -0.028106324, -0.005928678, 0.0026753321, -0.023068886, -0.021493072, -0.004559528, 0.009112597, -0.024024708, 0.003093504, 0.012296517, 0.023921376, -0.026608009, -0.015693558, 0.015422313, 0.0033163137, -0.040428676, -0.00016741008, -0.02250056, 0.011476318, -0.0013748009, -0.010765911, 0.024825532, -0.03482291, 0.024308871, 0.016106887, 0.06809584, -0.056367647, -0.018638523, -0.02994047, 0.0133298375, 0.0006296798, -0.006723043, -0.0046047354, -0.032807935, -0.007149288, -0.028390488, 0.03482291, 0.012005895, 0.012755053, -0.008544271, 0.01757937, -0.032187942, 0.012580681, 0.028700484, 0.0032258981, 0.014169411, 0.045982774, -0.0153706465, 0.0077886553, 0.032627102, 0.005473371, 0.018741855, 0.032704603, -0.012529015, 0.009196554, -0.006768251, -0.08028902, 0.021015162, -0.005880241, -0.018703105, -0.0013126403, 0.04528528, 0.0059609693, -0.00880906, -0.017476037, -0.010740078, 0.027383, 0.002076329, -0.012645263, -0.01591314, 0.013174839, -0.012800261, 0.013717333, 0.015254398, -0.026582176, -0.0006926478, 0.0098811295, -0.00832469, -0.0020117464, 0.030353798, -0.017230624, 0.0044174464, 0.028855482, 0.004927648, -0.0131683815, -0.010979033, 0.048824407, -0.017463122, 0.0072913696, 0.0037199548, -0.023365965, -0.043089475, 0.0046886927, -0.009810089, 0.0015637049, -0.02934631, -0.017256457, 0.040532008, -0.0068134586, 0.010165293, 0.015073567, -0.009370928, -0.03725121, 0.037509542, -0.007975944, -5.0127157e-05, -0.030069634, -0.022733057, -0.019581428, -0.028777983, -0.0077951136, 0.023081802, 0.0057704505, -0.060965925, 0.026892172, -0.0038329742, 0.0052118115, 0.027383, 0.03988618, 0.027512165, -0.0027512165, -0.030534629, -0.024283038, 0.02653051, 0.00013199059, 0.01946518, -0.005805971, -0.024037624, -0.010255708, -0.018483525, 0.02194515, -0.022952637, -0.0036198518, -0.004029951, -0.0030612128, 0.019026019, -0.019064767, -0.01809603, -0.008608853, -0.025187194, 0.011941313, 0.00610628, 0.03526207, -0.021828901, -0.012955259, 0.000919494, -0.027512165, 0.0056703477, -0.018987268, -0.00429151, -0.013077966, 0.03045713, 0.00028759416, -0.016920628, 0.007853238, 0.010023211, 0.0022297124, 0.036502056, -0.026168847, -0.022293895, -0.007601366, -0.006923249, -0.008118026, -0.028726317, 0.04339947, -0.020098088, 0.02272014, -0.012819636, -0.049315233, 0.012038187, 0.04156533, -0.0014902423, 0.0038814112, 0.018328527, -0.019529762, -0.02764133, 0.008647603, 0.008557187, -0.019064767, 0.012929426, -0.04766192, -0.02383096, 0.022087231, 0.0033066263, 0.0067617926, 0.017411456, 0.0034842284, -0.034616247, -0.0019084143, 0.013239422, 0.0032646477, -0.03688955, -0.001589538, 0.03918869, 0.01994309, -0.009661549, 0.020214338, 0.00091787946, -0.013704416, 0.0069297072, -0.015900223, -0.0032775642, 0.0101717515, -0.01672688, 0.005931907, -0.001838988, 0.002649499, 0.00067609857, -0.042469483, 0.002520334, 0.029656306, -0.027537998, -0.011185697, 0.006258049, 0.013510669, 0.0066907518, -0.021557655, 0.018289778, 0.007840321, 0.03714788, 0.026763007, -0.0037716208, 0.0059868023, 0.0046564015, -0.0018793521, 0.00910614, 0.0019665386, -0.0076724067, -0.022926804, -0.01424691, 0.023779294, 0.013975663, -0.0070072063, -0.017863533, 0.01424691, -0.010940284, 0.0037102674, -0.01591314, -0.0081826085, 0.01565481, 0.011353612, 0.012942342, 0.0009372542, -0.0049050446, -0.0014700602, -0.01480232, 0.008344065, 0.030276299, -0.008608853, -0.021867651, 0.015008984, -0.023508048, -0.03234294, -0.022293895, -0.037457876, -0.0060643014, -0.0088801, -0.040428676, -0.0024896571, -0.03988618, 0.025213027, -0.005302227, 0.002856163, -0.000345113, 0.019258516, 0.025471358, 0.026737174, -3.1408308e-05, 0.0017631035, -0.005111709, 0.037535377, 0.009532384, -0.03986035, -0.010475289, 0.0017388851, 0.009448427, 0.016933544, -0.008098652, 0.017514788, -0.022926804, -0.004327031, 0.026168847, 0.0056154523, -0.031877946, 0.01813478, -0.047455255, -0.0016726879, 0.00477265, 0.006748876, 0.012606514, 0.039472852, -0.022616807, 0.00041494286, -0.02498053, 0.020769747, -0.008570104, -0.026814673, -0.03696705, -0.016223136, 0.024903031, 0.054301005, 0.0050826464, 0.012167352, -0.0131167155, -0.018522274, 0.02375346, -0.0060804468, 0.018561024, -0.036372893, -0.0028448612, 0.018031448, 0.006361381, 0.022771806, 0.0015556321, -0.0058447206, -0.0034454789, 0.022410143, -0.011366528, 0.009299886, 0.000444005, 0.02079558, -0.0064485674, -0.02934631, 0.006383985, 0.006261278, -0.040506173, -0.011037157, 0.013207131, 0.0001873903, 0.01728229, 0.03063796, -0.0030918894, 0.012535472, -0.016029388, -0.021221826, -0.0020666416, -0.019762259, -0.026711341, -0.0073882435, 0.034926243, 0.005521808, 0.028002992, -0.012406307, -0.016145637, -0.01861269, 0.01565481, 0.00499546, -0.015435229, 0.006035239, 0.02131224, -0.014001496, -0.006109509, 0.03327293, 0.011812148, -0.030586295, -0.0019552365, 0.023327217, 0.033143762, 0.042727813, -0.024567202, -0.017191874, -0.008705728, -0.006923249, 0.029294644, -0.009222387, -0.009293429, 0.003987972, -0.006490546, 0.018522274, 0.013730249, 0.01960726, -0.016675213, 0.018896854, -0.006061072, 0.01960726, -0.023430549, -0.004214011, 0.004482029, -0.012806719, -0.0065841908, -0.030250465, -0.021828901, 0.0018551337, 0.027718829, -0.033841256, -0.03626956, 0.024528451, -0.0054346216, 0.026078433, -0.021493072, 0.029242977, -0.0077692806, -0.030947957, -0.039472852, -0.0057607633, -0.009784256, 0.0036456848, 0.03593373, -0.04673193, -0.027718829, -0.0043238015, 0.0074399095, -0.01561606, 0.016274802, 0.0034293332, 0.018767688, -0.011663608, 0.015189815, -0.027150502, 0.029656306, -0.0053603514, 0.008305316, 0.018418942, -0.012025271], "content_hash": "b535398474b391abdc4dcb7d8526648c"}, {"content": "Each appears as a chip (e.g., \"L0-H3\") with an \u00d7 button to remove it. 5. **Run the experiment**: Click \"Run Ablation Experiment.\" 6. **View results**: The panel shows a side-by-side comparison of the original vs. ablated generation, plus the probability change for the immediate next token. ### Understanding the Results\n\n- **Full Generation Comparison**: Shows the original generated text alongside what the model generates with those heads removed. If the text changed, the ablated heads were important for that generation.\n- **Probability Change**: Shows the immediate next-token probability before and after ablation (e.g., \"72.3% \u2192 45.1% (-27.2%)\"). A large drop means the head was important.\n- **No change?** If ablating a head has no effect, it may be redundant, or it may serve a function that isn't relevant to this specific prompt. Try a different prompt or a different head.\n\n### Tips\n\n- Start by ablating heads from the **Previous-Token** category -- these often have noticeable effects.\n- Try ablating heads from the **Other** category for comparison -- these often have less impact.\n- Use the **head categories** in the Attention stage (Stage 3) to pick interesting heads to ablate.\n- You can ablate heads from multiple layers simultaneously to see compound effects.", "source_file": "ablation_panel_guide.md", "chunk_index": 1, "embedding": [0.03795369, 0.019715818, 0.05737392, 0.0118088, -0.021297222, -0.02503643, 0.009015479, 0.026292684, 0.023750614, 0.038160603, 0.049304325, -0.030002333, -0.052053306, 0.026942981, -0.001459473, -0.024282675, 0.042594444, -0.028819975, 0.009200223, 0.04912697, -0.015991392, 0.003135096, 0.0049585137, 0.043392535, 0.036327947, -0.003935035, 0.031302925, -0.0010696645, 0.048949618, 0.015518447, -0.016833821, -0.018415224, -0.030194465, -0.003484261, -0.05793554, 0.0016562248, 0.01715897, 0.003467634, -0.0042490987, -0.017528456, 0.026750848, -0.026248345, -0.034702204, 0.028790416, -0.021385899, 0.042801358, 0.039490756, -0.05226022, -0.030682188, 0.049836386, -0.001244247, 0.07194648, -0.015917493, 0.0122374045, -0.036387064, -0.005313221, -0.00899331, -0.018060517, 0.015237638, -0.03322426, -0.016094847, 0.011084606, 0.022361344, 0.035293385, 0.000695559, -0.027652396, -0.037598982, 0.025893638, 0.009370186, 0.008091762, 0.034347497, -0.011934426, 2.3439323e-05, -0.0035249046, -0.031214248, -0.015503668, 0.025154665, -0.015503668, -0.0035249046, -0.013153732, 0.051875953, 0.012119168, 0.034140587, -0.02444525, -0.007929188, -0.036150593, -0.03431794, -0.018296989, -0.028864313, -0.075197965, -0.02558327, 0.029943215, -0.024622604, 0.01432131, -0.023277672, 0.049806826, 0.009377576, -0.009392356, 0.0040828297, 0.023248112, 0.03378588, -0.01574014, -0.025775403, -0.014262193, 0.0045964164, -0.011306298, -0.05867451, 0.0064734095, -0.030416157, 0.00873467, -0.012466487, -0.035382062, -0.02837659, 0.026750848, -0.005538608, -0.035263825, -0.029352035, -0.0028044053, 0.042742237, -0.041559882, -0.04495916, -0.0036098866, -0.009096766, 0.005708572, 0.0002279272, -0.0013671013, 0.05226022, -0.047087405, 0.019789716, -0.010981149, 0.01770581, 0.021385899, -0.019804494, -0.026809966, -0.045609456, 0.019021183, 0.01714419, 0.04924521, -0.07419296, 0.038810898, 0.0096066585, -0.023839291, -0.026603054, 0.010729898, -0.08016387, 0.011764461, 0.012850753, 0.060004666, -0.025080768, 0.052408013, -0.0020100086, -0.005254103, 0.00016084225, -0.014712966, -0.016124407, -0.04835844, -0.015562786, 0.0065103583, -0.03455441, -0.06603469, 0.029159902, -0.001665462, -0.018636918, -0.024489589, -0.019612363, 0.03322426, -0.0063847327, 0.00068586, -0.05648715, -0.02864262, -0.041796353, -0.036948685, -2.4435205e-05, -0.046555344, -0.055482145, -0.0016414453, -0.07046853, -0.019301994, -0.01770581, -0.028553944, 0.014978997, 0.030534394, -0.025760623, 0.048181087, -0.020380896, -0.050634477, 0.013249799, -0.008586874, -0.057787742, -0.023543702, 0.021681488, -0.0039165607, 0.06130526, -0.010478647, -0.012193066, -0.02809578, 0.084538594, 0.014749915, 0.004518824, 0.008365182, 0.014070059, 0.03479088, 0.0069241836, 0.0074968883, 0.0051543415, 0.0022169212, -0.011919646, 0.014565172, -0.0065583917, -0.0028561333, 0.044249747, -0.036682654, 0.01773537, 0.004012627, -0.04717608, -0.022420462, -0.03428838, 0.055334352, 0.029839758, 0.01486815, 0.03485, -0.009857909, 0.050398007, 0.07082324, -0.024386132, 0.031716753, 0.043481212, -0.016523452, 0.00077222753, -0.023942748, 0.032692198, 0.029248578, -0.020839058, -0.009776622, -0.022538697, -0.055836853, 0.029352035, -0.01859258, 0.031687193, -0.04859491, -0.033815436, -0.0066138147, 0.057699066, 0.009880078, 0.032130577, 0.033519845, -0.03505691, -0.069759116, 0.02468172, 0.038012806, 0.04886094, 0.008343013, -0.03597324, -0.0157697, 0.00632192, -0.0049548186, -0.015799258, 0.006909404, -0.0056088106, -0.010027873, -0.0053280005, -0.0008941582, -0.039963696, 0.02500687, -0.042890035, -0.008978531, -0.0022723442, -0.0015943358, -0.0516986, -0.03482044, -0.030046672, -0.010552544, 0.036889568, 0.018784711, -0.05143257, -0.0056790127, -0.013087224, 0.0027249656, -0.035648093, 0.031096013, -0.008343013, 0.005837892, 0.018326547, -0.0045040445, -0.039815903, 0.0023037505, 0.00024824898, 0.018459562, 0.010338242, -0.016671246, 0.012281743, -0.050782274, 0.08217388, -0.0224648, -0.032189693, 0.0064992737, 0.011904866, 0.02305598, -0.04779682, -0.00012805029, 0.024386132, -0.033253815, 0.03597324, -0.0030390294, 0.019893171, 0.014727746, 0.018370885, 0.022922965, -0.019922731, 0.0122374045, 0.007530142, 0.014801643, 0.007921798, -0.024356574, -0.029588507, -0.026809966, 0.03334249, -0.024193998, 0.0025679336, -0.022509139, 0.0019601278, -0.044811364, 0.009155884, 0.013382814, -0.016493892, 0.010286514, -0.00057547574, 0.0039239503, 0.011018098, -0.0081361, 0.03963855, -0.002948505, 0.018356107, 0.02051391, -0.017025955, 0.0004202913, -0.04522519, 0.047323875, 0.0147646945, 0.03514559, -0.056634944, 0.00252729, -0.0027249656, 0.00844647, -0.008003085, 0.026765628, 0.002667695, 0.018134415, 0.035352502, 0.02837659, -0.01910986, 0.0029244884, -0.012399979, -0.03730339, 0.054950085, 0.0680447, 0.006909404, -0.022361344, 0.043096945, 0.01265123, -0.0011186214, 0.014395208, 0.021297222, 0.02329245, 0.024371352, -0.028007103, -0.045905046, 0.027593277, 0.033312935, -0.014639069, -0.038958695, -0.020750381, 0.019228095, -0.00899331, -0.08282417, -0.006658153, -0.024341794, 0.003085215, -0.019745378, 0.034465734, 0.0054979646, -0.036121033, -0.041205175, -0.045402545, 0.04490004, 0.01431392, 0.0030741305, 0.010426919, -0.057492156, 0.017025955, 0.039195165, -0.009473643, -0.00021464877, -0.004851362, -0.02670651, 0.07732621, -0.04020017, 0.056605387, 0.02332201, 0.055511706, 0.004334081, 0.01628698, -0.045993723, -0.02389841, 0.046466667, 0.030209245, 0.017055513, -0.006713576, 0.030238803, 0.034909118, 0.05134389, 0.025997095, 0.03597324, -0.014070059, 0.022050975, -0.004832888, 0.043776803, 0.03006145, 0.02587886, 0.013079834, -0.028036661, -0.07638032, -0.017425, -0.04466357, -0.014698187, -0.020912956, -0.0047700754, -0.004101304, 0.045727693, -0.04948168, -0.039195165, 0.036150593, 0.040052373, -0.036948685, -0.0064032073, -0.040643554, 0.0048957006, 0.077089734, -0.042298853, -0.029086005, 0.015444551, 0.002024788, 0.02641092, -0.019139418, -0.07561179, 0.008076983, -0.014557782, -0.059147455, -0.013641454, -0.07348354, -0.03680089, -0.0018843829, -0.030889101, 0.06366997, 0.012991157, -0.06272409, 0.07626209, -0.005228239, 0.09648041, -0.00022654163, 0.021903181, -0.045609456, 0.0029300307, 0.048949618, -0.020898176, 0.05196463, 0.0065657813, -0.0084243, 0.03452485, 0.0021300917, 0.0005556159, -0.0047368216, -0.012015712, -0.023514144, -0.00660273, -0.007700106, 0.026632613, -0.007751834, 0.014195685, -0.052408013, 0.01630176, -0.026617832, 0.059029218, 0.00870511, -0.02638136, 0.09748541, 0.0009652844, -0.040407084, 0.020262659, -0.0026048822, 0.011350636, -0.020839058, 0.0162722, -0.0185778, 0.063315265, -0.01293943, 0.0037151903, 0.015873155, 0.021711048, -0.030209245, 0.035263825, 0.015444551, -0.003935035, 0.04776726, 0.0074599395, -0.02193274, 0.06916794, 0.05427023, -0.005261493, 0.056694064, -0.02638136, -0.033933673, -0.0047848546, -0.0007260417, -0.022982081, 0.01884383, 0.0013744911, 0.01884383, 0.020292219, 0.006410597, -0.006521443, 0.0075966497, -0.008062203, 0.012555163, -0.0023277672, -0.017262425, -0.02756372, -0.04525475, -0.041825913, -0.01770581, 0.028923431, -0.0325444, -0.004160422, 0.036357507, -0.042535327, -0.031214248, 0.011505821, 0.019228095, -0.039727226, 0.00344177, 0.022834288, 0.026218787, 0.022124873, -0.006654458, 0.01178663, 0.0020100086, 0.04835844, -0.01823787, 0.015828816, -0.007411906, 0.013700573, -0.0059007052, 0.025997095, -0.034731764, -0.020765161, -0.002301903, -0.026676951, -0.010057433, -0.00688354, -0.019760156, 0.021755386, -0.024696501, -0.0077370545, -0.020395674, -0.013375424, 0.025627607, 0.013168511, 0.0331947, 0.0061371764, -0.007378652, -0.00033900418, -0.014513443, 0.02982498, -0.040643554, 0.013744911, -0.0049843774, 0.0030999947, 0.022597816, 0.0026732374, 0.0016109627, 0.006802253, -0.027755853, -0.021755386, 0.03493868, -0.024223559, -0.014247413, 0.023824513, 0.012821194, -0.03647574, 0.0050287163, 0.016641688, -0.028790416, 0.041944146, 0.046850935, -0.021563252, 0.017927501, 0.03257396, -0.008823346, -0.013331085, -0.019065522, 0.010353021, 0.0076853265, -0.023987086, 0.03570721, -0.008756839, 0.02135634, -0.044988718, 0.00012354717, 0.0073306193, 0.000119967764, 0.010101771, -0.013944434, -0.053767726, -0.0009121707, -0.001240552, 0.029322477, 0.02021832, 0.006155651, 0.027120335, -0.027475042, 0.0012627214, 0.00034985787, -0.016449554, 0.0057603, 0.009865299, 0.01630176, -4.6258024e-06, -0.021445017, 0.018474342, 0.005095224, -0.04750123, 0.03452485, 0.01628698, 0.015126792, -0.010877693, 0.023750614, 0.024400912, 0.0038722223, 0.0001995229, -0.018607358, 0.00463706, -0.0030224025, 0.000555154, 0.0027748463, -0.046850935, 0.019568024, 0.017794486, -0.0054684053, -0.019139418, -0.02555371, 0.006750525, -0.017262425, 0.057462595, 0.004851362, 0.0010530375, -0.03647574, 0.00259934, -0.019656701, -0.003131401, -0.011609277, -0.014291751, 0.021267664, -0.03768766, -0.030682188, 0.0033069074, 0.025908418, 0.0016423691, 0.007951357, -0.013035496, 0.0036690044, -0.033017345, 0.016626908, 0.033667643, 0.010545155, -0.011550159, 0.048713148, -0.033549406, -0.03485, -0.01630176, 0.018311769, -0.008025254, 0.0058637564, -0.032455724, -0.007053504, 0.011601887, 0.0032681113, 0.017883163, -0.010877693, -0.000369025, 0.0035415315, -0.018208312, -0.009673166, 0.003262569, -0.009340628, -0.012200456, 0.0021467186, -0.016050508, 0.044338424, -0.0020469571, -0.054122433, -0.0010881388, -0.022568258, 0.020853838, -0.0031849768, -0.0021134648, 8.4520114e-05, 0.0050841393, 0.05370861, 0.027282909, -0.01236303, -0.010803795, 0.015518447, -0.0097175045, 0.01970104, -0.031066455, 0.03520471, -0.016390437, -0.017336324, -0.008313454, -0.050723154, -0.008468639, 0.023425467, -0.022272667, -0.011040268, 0.006802253, -0.010035262, 0.0055792513, -0.03875178, -0.024888635, -0.0062517175, 0.012030492, 0.021873621, 0.0065879505, -0.028494826, -0.0012137643, 0.014225244, -0.0155332275, -0.011055047, 0.0055718618, -0.021223325, 0.012525604, -0.014151347, 0.018932506, 0.019257655, -0.000927412, -0.0009130944, -0.00788485, 0.016198304, -0.02024788, -0.025967536, -0.021001633, 0.014971607, -0.004947429, -0.005560777, 0.0275785, 0.015067674, -0.0010798253, -0.01993751, -0.0039128656, 0.030829983, 0.018282209, -0.0049326494, 0.007197604, 0.02364716, -0.0076631573, -0.019257655, 0.013722742, 0.00954754, 0.010478647, 0.02330723, -0.010234786, 0.008062203, 0.041619, 0.0033069074, 0.032987785, -0.031332485, 0.050723154, -0.013382814, -0.034968235, 0.011010708, -0.015503668, -8.919644e-05, -0.063847326, 0.0044264523, -0.016686026, -0.020085305, -0.01265123, -0.021297222, -0.021178987, -0.027489822, 0.0124590965, 0.014424766, 0.009473643, -0.027061217, -0.000398353, 0.0018363496, -0.0043045217, -0.10942723, 0.003523057, 0.013050276, -0.041559882, -0.016745145, -0.019715818, 0.019050742, 0.0027009489, 0.015932273, 0.0021430238, 0.008483418, -0.0147646945, -0.0021614982, -0.030800425, 0.028893871, -0.044574894, 0.012473877, -0.031243809, 0.03703736, -0.0465849, -0.027016878, 0.008535147, -0.0023092928, -0.0146612385, -0.054595377, 0.007190214, -0.037628543, 0.0008973912, -0.00591179, 0.022065755, -0.021548472, 0.012252184, -0.01939067, 0.018666476, -0.024001867, 0.0037004107, -0.009444084, -0.008003085, 0.004552078, 0.011350636, 0.03399279, -0.029337255, 0.0056974874, -0.004467096, 0.0034657866, -0.017897943, -0.020528689, -0.0021319392, 0.026588274, -0.0013532456, 0.00462967, -0.000829498, 0.0058822306, 0.038042367, 0.014188295, -0.03851531, 0.0005505354, -0.0019545855, 0.0052171545, 0.023854071, 0.026307464, 0.05920657, 0.016050508, -0.011697954, 0.0065140533, -0.014587341, -0.01208222, 0.017425, -0.038042367, -0.048269764, -0.029026886, 0.028982548, -0.010256955, 0.006717271, 0.03597324, 0.030150127, 0.04014105, 0.03091866, 0.0022704967, 0.00869772, 0.02555371, -0.039076928, -0.011106775, 0.019301994, -0.0068983193, 0.028287914, 0.02332201, 0.0025291375, 0.0100500425, 0.034199703, 0.031066455, -0.042564884, -0.0032052984, 0.04306739, -0.051048305, -0.020912956, 0.02979542, 0.015503668, 0.016819041, 0.0055496925, -0.004980683, -0.00086506107, -0.007530142, -0.030889101, -0.015488889, 0.015208079, 0.0029595897, 0.014816423, -0.017085072, -0.007448855, 0.032987785, 0.0155332275, -0.019257655, 0.040318407, -0.021873621, 0.022819508, 0.029086005, -0.018385665, -0.0185778, -0.03148028, -0.00013359259, 0.038929135, -0.012887701, -0.007563396, 0.0056272848, 0.027977545, -0.014247413, 0.008535147, -0.0026566104, 0.016730364, 0.0145430025, -0.0071865194, -0.029721523, -0.010818575, -0.0064512403, -0.011010708, -0.0034306855, 0.00079347304, -0.009207612, 0.014018331, -0.02553893, -0.0014419224, 0.03118469, -0.009118935, -0.0556595, -0.038929135, -0.00281549, -0.012266964, -0.031923663, -0.0069537424, 0.011424533, 0.01856302, 0.003526752, -0.00023878088, 0.009917027, 0.015681023, 0.04522519, -0.008867685, -0.0058600614, 0.048240203, 0.010234786, 0.021415457, 0.034909118, 0.010574713, -0.00017885472, 0.0041973707, -0.03508647, 0.0036911736, -0.012525604, 0.006521443, -0.03207146, 0.0041493373, 0.004962208, 0.011010708, 0.034436174, 0.019035963, -0.008897243, -0.012392589, -0.012828584, 0.028125338, -0.006828117, 0.0155332275, -0.01885861, -0.021563252, -0.0030408767, -0.016523452, 0.018888168, 0.021149427, 0.022583036, -0.02134156, -0.0092889, 0.026292684, -0.0032958228, 0.031864546, -0.020011408, -0.011927036, 0.017439779, 0.013981382, -0.055600382, -0.035500295, -0.04634843, 0.066744104, 0.015681023, -0.007267806, 0.026100552, 0.028184457, -0.03313558, 0.010183058, 0.005985687, 0.023750614, -0.028982548, 0.019789716, -0.006092838, 0.00505458, -0.019553244, 0.007833121, -0.022154432, 0.00060688215, 0.013138953, -0.012045272, -0.048269764, -0.037273835, -0.015961831, 0.033963233, -0.012156118, -0.0023536312, 0.026529156, 0.004973293, 0.0073306193, 0.016656468, -0.031657632, -0.026928201, 0.0034824135, -0.0010151651, -0.022893405, 0.046259753, -0.025420696, -0.011624056, -0.0063995123, 0.012540384, 0.037746776, -0.01234086, -0.02839137, -0.021267664, -0.008121321, -0.020440012, 0.022612596, 0.01435087, 0.0071458756, -0.031302925, -0.021193765, -0.016922498, 0.02414966, -0.019981848, 0.018947287, -0.0040311017, 0.039993256, 0.016626908, 0.039756786, 0.022021417, -0.035766326, 0.024888635, 0.010005704, -0.044781808, 0.007563396, 0.011971374, -0.024327014, -0.013013327, 0.03875178, -0.018208312, 0.029026886, 0.052378457, 0.007005471, 0.021105088, -0.0019638226, -0.010818575, 0.017321544, -0.0022741915, 0.015030725, -0.009229781, 0.034909118, 0.027652396, -0.014794254, 0.008490808, -0.0058046384, -0.023735836, -0.00702764, -0.009650997, 0.0027840834, 0.0016885549, -0.00547949, 0.019597583, 0.009333238, 0.012407368, -0.00591179, 0.021755386, 0.0068724556, 0.013848367, 0.018755153, -0.052614927, 0.0032958228, -0.00047386688, 0.03349029, -0.03898825, -0.0185778, 0.0022520223, 0.032455724, -0.05001374, 0.004688788, 0.0049511236, -0.009665776, 0.04218062, -0.0052688825, 0.017336324, -0.02555371, -0.016715584, 0.012725127, -0.013966603, -0.004496655, -0.013641454, -0.023395907, 0.018178754, -0.00067292794, 0.030534394, -0.008845516, -0.02979542, 0.0010207074, -0.0036948684, -0.002723118, -3.022287e-05, 0.016804261, 0.0015998781, -0.014358259, 0.007833121, 0.0015167436, 0.020144423, -0.031036895, 0.0037872402, 0.008971141, -0.0126807885, -0.027608057, 0.026691731, -0.026676951, 0.0058674514, 0.013331085, 0.00085536204, -0.0055718618, -0.02191796, 0.023780175, 0.003011318, 0.08442035, -0.05873363, -0.03405191, -0.028494826, 0.004851362, 0.014395208, -0.0019379585, -0.01770581, -0.052969635, -0.014151347, -0.03260352, 0.02049913, 0.007175435, 0.017203309, 0.009030258, 0.020912956, -0.026529156, 0.0035489213, 0.034613527, -0.00239058, 0.017380662, 0.0404662, -0.022612596, 0.012606892, 0.017070293, 0.011491041, 0.012784245, 0.033608522, 0.009081987, 0.02165193, -0.014306531, -0.042210177, -0.0056088106, -0.016331319, -0.025893638, -0.0052356287, 0.011003318, 0.018651696, -0.024208779, -0.023100318, -0.02558327, 0.023794953, 0.016700806, -0.00758187, 0.009333238, -0.012259574, 0.0039165607, 0.002752677, 0.005257798, -0.025139885, -0.014860761, -0.00591179, -0.02055825, -0.012496046, 0.03993414, -0.04218062, 0.0031942138, 0.02836181, -0.0010456478, -0.008224777, -0.010988539, 0.035648093, -0.014136567, 0.0033124497, -0.013308916, -0.047915056, -0.025923198, 0.024666943, -0.003820494, 0.015089843, -0.028923431, -0.020129643, 0.029248578, -0.013464101, 0.007249332, -0.0016756229, -0.018503902, -0.029647624, 0.039195165, 0.009229781, 0.0030611986, -0.03742163, -0.030948218, -0.02160759, -0.0133384755, -0.011978764, 0.026647393, 0.011638836, -0.03544118, -0.0055127437, -0.018666476, 0.011594498, 0.055157, 0.04977727, 0.0089415815, -0.021415457, -0.019981848, -0.014025721, 0.013249799, -0.02245002, 0.013316306, -0.011077216, -0.010367801, -0.013737521, -0.020114865, -0.017173748, -0.044515777, 0.024873855, 0.008579485, 0.01152799, -5.9868416e-05, -0.008963751, -0.004496655, 0.012525604, -0.008246946, 0.003493498, -0.0049548186, 0.058112893, -0.025346799, -0.040613994, 0.010766847, -0.0030094704, 0.0037447494, 0.003063046, 0.0064771045, -0.012303912, 0.022878626, -0.0013320001, 0.0019379585, -0.00450035, 0.011195452, -0.0024903414, 0.023809733, -0.003258874, -0.014949438, -0.01712941, -0.01630176, 0.0011934425, -0.017927501, 0.029322477, -0.006325615, 0.01967148, -0.02107553, -0.04832888, 0.007323229, 0.030889101, 0.0157697, -0.00081748964, 0.010619052, -0.02553893, -0.034170143, -0.00052928994, 0.02160759, 0.0029577422, 0.010330852, -0.025139885, -0.023011642, 0.0027009489, 0.027327247, 0.012318691, 0.010212617, 0.0013707962, -0.021696268, -0.005368644, 0.0057713846, -0.011084606, -0.034702204, -0.0054942695, 0.036150593, -0.00589701, -0.009429305, 0.009739674, -0.014188295, -0.007810952, 0.014070059, -0.017750148, 0.0038020196, 0.039461195, -0.0056974874, 0.021297222, -0.009798791, -0.012326081, 0.0024663247, -0.018814271, -0.01040475, 0.016922498, -0.018784711, -0.0043710293, -0.0065694763, 0.013678404, 0.012126558, -0.024193998, 0.0010825965, 0.0086164335, 0.023277672, 0.036357507, -0.010818575, 0.012828584, 0.031864546, -0.012466487, 0.005542303, -0.010626442, -0.009089377, -0.012399979, -0.02698732, 0.027312469, 0.004330386, -0.018282209, -0.001982297, 0.0057898588, 0.004703568, 0.004814414, -0.0074082115, -0.009074597, 0.01964192, -0.0149863865, 0.00898592, -0.007190214, 0.0005362178, -0.008764228, -0.01628698, 0.017617133, 0.03913605, 0.02363238, -0.024844296, 0.00047525245, -0.008165659, -0.011077216, -0.015902715, -0.007057199, 0.010072212, -0.02414966, -0.030297922, 0.0040015425, 0.0020728211, 0.020040967, -0.031421162, -0.019981848, 0.0090598175, 0.020765161, 0.010227396, 0.0208095, -0.013959213, 0.022612596, -0.023248112, 0.057758186, -0.0032902805, -0.014890321, -0.015858376, -0.008719889, 0.031716753, 0.006913099, -0.020750381, 0.00083781144, -0.031243809, -0.02781497, 0.008217388, -0.0041715065, -0.026337022, 0.0059709074, -0.05370861, -0.008527757, -0.011151114, -0.007892239, 0.011217621, 0.027223792, -0.0016395979, -0.0009015479, -0.031568956, 0.058910985, -0.012946819, -0.0015546159, -0.019154198, -0.00788485, 0.017691031, 0.03505691, 0.00449296, 0.012666009, -0.008904633, -0.023957528, 0.003820494, -0.029352035, 0.007833121, -0.01179402, -0.0071532656, 0.02836181, 0.014107008, 0.013840977, 0.017173748, -0.011734903, -0.0030999947, 0.03201234, 0.0041973707, 0.015060284, -0.022996861, 0.0036930211, 0.006635984, -0.02391319, -0.012991157, -0.0101534985, -0.0371556, -0.014143957, 0.009488422, -0.007951357, 0.010515596, 0.025819741, -0.01378186, 0.02417922, 0.003347551, -0.018356107, -0.0072012986, -0.023558483, -0.011904866, 0.008091762, 0.021415457, 0.028406149, 0.017957062, -0.015858376, -0.039431635, -0.010301294, -0.0112471795, 0.023100318, -0.023543702, 0.0012525604, 0.0017439779, -0.0123482505, -0.021504134, 0.03594368, 0.0024829516, -0.022036197, -0.004829193, 0.036889568, 0.032160137, -0.001763376, -0.0048957006, -0.011956595, 0.0024847991, -0.027785411, 0.0224648, -0.019848833, -0.022095313, -0.0013079834, 0.00843908, 0.002854286, 0.012702958, 0.032189693, -0.008793787, 0.02809578, -0.018932506, 0.002314835, 0.0018853067, -0.0014557781, 0.0028616756, 0.006351479, 0.004515129, -0.0044818753, -0.0006932497, -0.011978764, 0.008771618, -0.054595377, -0.031864546, -0.0149863865, 0.012577333, 0.026883863, -0.016656468, 0.019819275, -0.007618819, -0.033549406, -0.0208095, 0.005667928, -0.02614489, 0.01856302, 0.027962765, -0.019346332, -0.014528223, -0.0073195347, -0.00899331, -0.026499597, 0.009998314, -0.028347032, 0.029189462, -0.021548472, 0.006096533, -0.01909508, 0.03597324, 0.0014816422, -0.00046255134, 0.021312002, -0.01207483], "content_hash": "65f446fd27f72a314995f8ed45a20c90"}, {"content": "# The Attention Mechanism\n\n## What Is Attention?\n\nAttention is the core innovation that makes Transformers powerful. It allows the model to look at **all tokens in the input simultaneously** and figure out which ones are relevant to each other. When processing any token, the model can \"attend to\" (focus on) other tokens that provide useful context.\n\n## An Intuitive Analogy\n\nImagine you're reading a sentence and you come across the word \"it.\" To understand what \"it\" refers to, you look back at earlier words -- maybe \"the cat\" or \"the ball.\" Your brain is doing something like attention: scanning the context and focusing on the most relevant parts.\n\nThe model does this mathematically by computing **attention scores** between every pair of tokens.\n\n## How It Works: Queries, Keys, and Values\n\nAttention uses three concepts (don't worry about the math -- the intuition is what matters):\n\n- **Query (Q)**: \"What am I looking for?\" -- each token asks a question\n- **Key (K)**: \"What do I contain?\" -- each token advertises what information it has\n- **Value (V)**: \"Here's my actual information\" -- the content that gets passed along\n\nThe model computes how well each Query matches each Key. High matches mean strong attention. The final output for each token is a weighted combination of all the Values, weighted by these attention scores.\n\n## Multi-Head Attention\n\nInstead of computing attention once, Transformers compute it multiple times in parallel using different **attention heads**. Each head learns to look for different kinds of relationships:\n\n- One head might track which word a pronoun refers to\n- Another might focus on the immediately preceding token\n- Another might spread attention broadly across the sentence\n\nGPT-2 has **12 attention heads per layer** (144 heads total across 12 layers). The dashboard categorizes these heads to help you understand their roles.\n\n## What You See in the Dashboard\n\nIn **Stage 3 (Attention)** of the pipeline, you can see:", "source_file": "attention_mechanism.md", "chunk_index": 0, "embedding": [0.009518963, 0.054300353, 0.027301572, 0.02299943, 0.010200241, -0.040523406, 0.0064973705, 0.04413165, -0.03320598, 0.036460973, 0.024690008, -0.035375975, -0.02717541, -0.018293567, -0.012395469, -0.029471567, -0.007986088, -0.009323411, 0.033862025, 0.029622963, -0.031565864, -0.0015044882, 0.04771467, 0.055158257, 0.012849654, 0.0005424989, -0.032575164, 0.029042615, 0.014988109, -0.013512007, 0.039943058, -0.018369265, -0.048496876, -0.008743064, 0.022229837, 0.029622963, -0.021573793, 0.03191912, -0.021409782, 0.020627573, 0.020400481, 0.0147484, -0.036814228, 0.053089194, -0.0274782, -0.0033243827, 0.016022641, -0.018104324, -0.045822233, 0.07842263, -0.011733116, 0.047336183, -0.0416841, 0.034543302, 0.040018756, 0.02417274, -0.048421178, 0.03585539, 0.0122882305, -0.012792881, -0.010446258, -0.0074372813, 0.037697364, 0.039034687, -0.0115943365, 0.00025311357, -0.050061293, 0.008944923, -0.021813503, -0.01582078, 0.05054071, 0.0579843, -0.00072267477, 0.006938939, 0.039186083, -0.015215201, -0.02535867, 0.033962954, -0.044030722, -0.0070083286, 0.024462914, -0.042668168, 0.019857982, -0.029168777, -0.02485402, -0.032650862, -0.032877956, -0.067068, -0.04039724, -0.052534077, -0.018091707, 0.028260406, -0.014483458, 0.004131823, 0.014861946, 0.041431773, 0.004368378, 0.010723815, -0.010698583, 0.022848034, 0.062374756, 0.0056583895, -0.013032589, -0.0173095, 0.031364005, -0.018798217, 0.028411802, -0.0025122114, -0.008881842, -0.037293643, -0.07287148, -0.013688635, -0.008263646, 0.016010026, 0.02616611, -0.009588352, -0.00194448, 0.013940959, -0.00025311357, -0.010767972, -0.03272656, -0.023756405, 0.044156887, 0.028058548, 0.01286227, -0.01245855, 0.033483535, -0.01606049, 0.005570076, 0.023340069, 0.003955195, 0.019479495, 0.020097692, -0.022608325, 0.010938291, -0.04299619, -0.022419082, 0.025762388, -0.042542007, 0.0020217546, 0.010093003, -0.018760368, -0.00012330415, 0.009821753, -0.04299619, 0.0024302057, -0.018810833, 0.054148957, -0.039615035, 0.016552525, -0.0006990193, -0.02128362, -0.069894046, -0.039640266, -0.0051600477, -0.011039222, -0.010799512, 0.06903613, 0.002042256, -0.023289604, 0.036385275, 0.0012395469, -0.0114114005, -0.029572498, -0.041532706, 0.045872696, -0.02299943, 0.0032234527, -0.024425065, -0.0559657, -0.039514106, -0.026292272, 0.0041255145, -0.011070762, 0.04236538, 0.00398989, -0.06666428, -0.02053926, -0.0007605235, -0.033306908, 0.0035672456, -0.016527291, -0.014369912, 0.019580426, -0.01952996, -0.011480791, -0.019277634, 0.04150747, -0.026267039, 0.026418434, -0.03131354, 0.03835341, 0.04128038, 0.028411802, -0.019921064, -0.030783657, 0.036107715, 0.01942903, 0.047033392, 0.0066550737, 0.040043987, 0.02990052, 0.071559384, -0.022469547, 0.054552678, 0.03643574, -0.005683622, -0.002745612, -0.054401282, -0.043046657, 0.024059195, -0.07044916, 0.028765056, 0.013587705, -0.03161633, -0.012376544, -0.062273826, 0.0549564, 0.0077526877, 0.041835494, 0.02047618, 0.012786573, 0.014319447, 0.033407837, -0.045923162, 0.04657921, 0.03623388, -0.03565353, 0.015543224, -0.022204606, 0.017233802, 0.010332711, 0.011367244, -0.018861298, -0.04801746, -0.056419883, 0.03883283, -0.031439703, 0.056722675, -0.02959773, -0.015467526, -0.009518963, -0.013852646, 0.029622963, 0.014420377, 0.021561177, -0.010887827, 0.00017298065, 0.06070941, 0.041028056, 0.048295017, 0.003292842, -0.06267755, 0.007967164, -0.019681355, 0.025106344, -0.014899795, 0.058287088, 0.0024507071, 0.038126316, -0.014597005, 0.023491463, -0.03797492, -0.045140952, -0.012174685, -0.01680485, -0.0012758186, -0.009216173, -0.05344245, -0.031565864, -0.01767537, 0.026519364, 0.017347349, 0.0040624337, -0.046049323, -0.029168777, -0.002786615, 0.01585863, -0.0042579854, 0.002042256, -0.014041889, -0.011013989, 0.049581874, 0.031944353, -0.007178648, -0.00038775266, -0.013915727, -0.012893811, -0.0035735536, -0.01626235, 0.035073183, -0.038630966, 0.064090565, -0.0017173875, -0.007954547, 8.422335e-05, 0.0044535375, 0.01706979, 0.026620295, -0.010427333, 0.035577834, -0.038630966, -0.0011654263, -0.011657418, 0.0014327333, 0.017423045, -0.01505119, -0.008269954, 0.0071471073, -0.018558508, 0.008534895, -0.03164156, 0.011354628, -0.011310471, -0.009449573, 0.0065036784, 0.03694039, 0.01293166, -0.014130203, 0.013057822, -0.036057252, -0.035880625, 0.00013424482, 0.004084512, 0.03827771, -0.009165708, -0.031187378, 0.011417709, -0.007721147, -0.039110385, 0.07388078, -0.025068495, 0.00045970472, -0.0249171, -0.024866635, -0.0056142327, 0.015543224, 0.045544673, -0.026948316, 0.027730525, -0.020286934, -0.0109004425, -0.06333359, -0.018962229, -0.052534077, 0.05324059, 0.019353332, -0.012363928, 0.012408085, -0.011329396, -0.0091467835, -0.023668092, 0.025812853, -0.04006922, -0.00956312, 0.025093727, -0.030960286, -0.009014313, 0.00029313075, -0.01791508, -0.01579555, 0.004223291, 0.04912769, -0.000117291725, -0.0021731495, -0.025030646, -0.03532551, -0.0009367568, -0.010818437, -0.001696886, 0.037217945, 0.050666872, 0.01595956, -0.014496075, -0.016186653, -0.011070762, 0.017587056, -0.0028197325, -0.030329473, 0.021763036, 0.022482162, 0.023390533, -0.033988185, -0.020085076, 0.033534, 0.039640266, -0.0008216335, -0.001696886, -0.066361494, -0.019101007, -0.0264689, -0.040321544, -0.025446983, 0.019012693, 0.008490738, 0.03482086, 0.019542577, 0.037899226, 0.04319805, 0.057126395, 0.031263076, 0.017952928, -0.041936424, -0.013537239, 0.025711924, -0.0012285076, -0.009582045, 0.011827737, 0.01807909, 0.008610593, 0.0752938, 0.007613909, 0.01868467, 0.007916699, -0.011285238, 0.018243102, 0.017713219, 0.0064342893, 0.012231457, 0.025081111, -0.051676173, -0.03191912, -0.038403876, 0.011323087, -0.07080241, 0.016022641, 0.057631046, -0.039211314, 0.024500763, -0.027024014, 0.0024775167, 0.047411878, 0.010244397, -0.003195066, 0.011083378, -0.0146600865, 0.048093155, 0.037394576, 0.010597653, -0.037571203, 0.026267039, -0.055915233, -0.018848682, -0.027856687, -0.03484609, 0.010383177, -0.025560528, -0.051474314, -0.0007605235, -0.033811558, 0.02299943, 0.047462344, 0.024298903, 0.020324783, -0.03777306, -0.0057088546, 0.04559514, 0.041154217, 0.028386569, -0.019555192, 0.017233802, -0.0076454496, -0.016035257, 0.007866234, -0.017498743, -0.021258386, 0.0015107963, -0.032348074, 0.03090982, -0.009317103, 0.0022709256, -0.005529073, 0.00204541, -0.021573793, 0.041431773, -0.031364005, 0.028159477, 0.010849978, -0.002039102, -0.05889267, 0.00516951, -0.009600969, -0.025093727, 0.02255786, 0.020577108, 0.061769176, -0.009689283, -0.021649491, 0.028008081, -0.031540632, -0.010711199, -0.03270133, 0.026519364, -0.012704567, 0.020425715, -0.01293166, -0.047033392, -0.037091784, -0.00986591, -0.013045206, 0.034770392, -0.005626849, 0.03302935, 0.074334964, -0.0028843908, 0.025573146, 0.049884666, 0.036713295, -0.0146979345, 0.024841402, -0.021081759, -0.009783904, 0.02532082, 0.032272376, -0.023832101, 0.017195953, 0.042314913, 0.02094298, 0.007853618, 0.0025579452, -0.0070903343, 7.914728e-05, -0.0011417709, -0.0025232506, 0.0037722597, -0.008604284, -0.050389316, -0.016994093, -0.06515033, -0.030834123, 0.0034789317, -0.018949613, 0.026695991, 0.018520659, -0.039211314, 0.010219165, 0.0021747267, -0.045973625, -0.060658947, -5.5639648e-05, 0.0026226037, 0.007134491, 0.02047618, 0.011941284, 0.015669387, 0.008068094, 0.021510711, -0.03471993, 0.0032187216, 0.049405247, 0.0012537402, -0.034063883, 0.012370236, -0.021018678, -0.023819486, -0.013524624, -0.009884834, -0.0118845105, 0.009727132, -0.025787622, 0.0136381695, 0.0013420539, -0.0036618675, -0.035905857, -0.03209575, 0.04017015, 0.009626201, -0.0064122104, 0.0004297411, -0.033559233, -0.013877878, -0.020816818, 0.011493406, -0.015896479, 0.020350017, 0.008055477, -0.0026005253, 0.024702623, -0.0074120485, 0.028310873, -0.032549933, -0.017044557, -0.034997486, -0.0046080863, -0.024248438, 0.004929801, -0.011991749, -0.011543872, -0.028437035, -0.004384148, 0.02511896, -0.029017381, 0.012029598, 0.020450946, -0.025888551, -0.015013341, 0.009077394, 0.005194742, -0.03000145, 0.03310505, 0.010029921, 0.022671407, -0.0070587937, -0.017839381, -0.017359965, 0.017687988, -0.024929715, 0.0012671449, 0.025282972, 0.011386168, -0.03636004, -0.034089115, -0.04201212, 0.00087130995, -0.05324059, 0.030834123, -0.037545968, 0.02969866, 0.004037201, 0.016047874, 0.015618921, -0.017952928, 0.018747753, 0.0064342893, 0.00956312, 0.0315154, 0.010736431, -0.039211314, -0.00909001, 0.014634853, -0.053038727, 0.012666718, 0.04049817, 0.00868629, -0.026014714, 0.020009378, 0.0013318033, -0.01414282, 0.04097759, -0.015139503, -0.021977512, 0.0074120485, -0.025446983, 0.011291547, -0.0050622714, 0.009455882, -0.01026963, 0.010263322, -0.016880546, -0.02017339, -0.036057252, -0.046680138, 0.067068, -0.00020166292, 0.0109004425, -0.008881842, -0.048774436, -0.008976464, 0.0014114433, 0.017826766, -0.040927123, 0.0031651026, -0.009619893, -0.03986736, -0.01966874, 0.009481114, 0.018508043, 0.018470194, 0.022633558, 0.009708207, -0.090635166, 0.03933748, -0.011562796, 0.02195228, 0.0249171, 0.010433641, 0.020097692, -0.043753166, -0.00378803, 0.049304318, -0.0036524052, -0.004440921, -0.04198689, -0.007128183, 0.021422397, 0.005532227, -0.0045765457, -0.023844719, 0.05172664, -0.0064185187, -0.004223291, -0.007191264, 0.023554545, -0.034164812, -0.027730525, -0.029269706, 0.015871247, 0.063585915, -0.017473511, -0.05980104, -0.019769669, -0.0039078845, -0.03164156, 0.011695267, -0.01811694, -0.01585863, 0.025888551, 0.0722659, 0.008989081, -0.041936424, -0.044838164, 0.030152844, 0.02255786, 0.024286287, 0.0056079244, 0.05023792, -0.033054583, -0.015328747, 0.0074498975, -0.018508043, -0.02518204, 0.023756405, -0.0067938524, 0.0027345729, 0.01993368, -0.016539907, -0.030581798, -0.008843994, -0.008364576, 0.027730525, 0.006491062, 0.007885158, -0.015013341, -0.07377985, -0.022343384, 0.022027979, -0.029446336, -0.0124333175, -0.03696562, -0.023491463, 0.0033496153, -0.005131661, 0.01384003, 0.023340069, 0.02154856, 0.025939016, -0.0030436711, 0.009878526, -0.037091784, -0.033887256, -0.013713867, 0.01006777, 0.016438978, 0.0013538817, -0.0073742, 0.002687262, 0.029421102, -0.014861946, 0.033407837, 0.0294968, -0.008471814, -0.033534, -0.043298982, 0.025838086, 0.0025153654, -0.037495505, 0.0021163765, -0.012710875, 0.032348074, 0.015139503, 0.0020469872, 0.0031714106, 0.029067848, -0.008944923, 0.007626525, -0.020968212, 0.042138286, -0.00983437, 0.0053587537, -0.017662754, 0.022267686, 0.0017851999, -0.022027979, -0.039917827, 0.0033212285, -0.009720823, -0.025989482, -0.016855314, -0.013171368, -0.0031635254, -0.02306251, 0.0008015263, 0.023075126, -0.026569828, -0.006118883, 0.008446582, 0.004727941, -0.07741333, -0.04044771, 0.050591175, -0.048295017, -0.0119223595, -0.03333214, 0.035047952, 0.0081501, 0.021069143, -0.02616611, 0.0037091784, -0.026822153, 0.008806145, -0.004825717, 0.0315154, -0.035451673, -0.010887827, 0.0036208646, 0.016237117, -0.002486979, -0.037621666, -0.011436634, 0.017486127, -0.011209541, -0.024828786, 0.014016657, -0.006131499, 0.012773956, 0.005825555, -0.0012852808, -0.02919401, 0.016325431, 0.0051064286, 0.04723525, 0.0036965623, 0.026796922, -0.0040182765, 0.043046657, 0.017359965, -0.015341364, -0.002999514, -0.024690008, 0.013751715, -0.030178078, 0.010730123, 0.01178358, 0.01178358, -0.019958913, 0.00617881, 0.035022717, -0.025636226, 0.01983275, -0.015063806, 0.014609621, 0.031061215, 0.02501803, -0.019757053, 0.0033780017, 0.009228789, -0.011871894, -0.023075126, -0.00035522637, 0.056318954, 0.007443589, 0.034013417, 0.010414717, 0.013587705, 0.032575164, -0.01862159, -0.024702623, -0.0006891628, 0.0032281838, 0.009828061, -0.011342011, -0.013877878, -0.027276339, 0.0038511113, 0.026494132, 0.0099857645, -0.021081759, 0.06232429, -0.014407761, -0.02094298, 0.021561177, 0.013575088, 0.02100606, -0.017233802, -0.010477798, 0.032171447, 0.03734411, -0.0035987862, -0.0029821668, 0.026267039, -0.0133479955, 0.008433965, 0.00818164, 0.022280304, 0.04186073, 0.07953286, 0.0039488873, 0.0610122, 0.01942903, -0.0042611393, -0.043021422, -0.053492915, 0.011386168, -0.02798285, -0.01761229, -0.0019097853, -0.015290899, -0.0017804687, 0.00044905976, 0.01888653, 0.0069957124, 0.015896479, 0.035905857, 0.035804927, 0.012313463, 0.023188673, -0.009746055, 0.0071597234, 0.0068190848, -0.03696562, -0.003191912, -0.013310147, -0.018873915, -0.0013065707, 0.033660162, 0.037091784, -0.011127535, 0.0056079244, 0.029925752, 0.019580426, -0.019794902, -0.012527939, 0.017006708, 0.026039947, 0.026569828, -0.011852969, 0.031767726, -0.039615035, -0.0048004845, 0.015290899, -0.0011938129, -0.017751068, -0.01791508, -0.0054439134, -0.043298982, -0.025674075, 0.021056527, 0.03310505, 0.005229437, 0.017246418, 0.04347561, -0.010143467, 0.03424051, -0.010093003, -0.042062588, -0.02205321, 0.027907152, 0.0078031523, 0.033407837, 0.061062664, 0.0073678917, -0.00051490084, -0.00609365, -0.020993445, 0.008610593, -0.019340716, 0.016956244, -0.029042615, 0.014268982, 0.04869874, 0.029875288, 0.025156809, -0.003138293, -0.010282246, -0.0016795387, 0.022116292, 0.021106992, -0.016968861, 0.024841402, 0.004980266, -0.0036555594, 0.023491463, 0.014218518, -0.010805821, 0.018356647, -0.011802505, -0.034997486, 0.024626926, 0.007084026, -0.022343384, 0.0023734325, -0.0336097, -0.021258386, 0.020993445, 0.0037154865, -0.00959466, 0.016552525, -0.0012356043, 0.026897851, 0.010919367, -0.015013341, -0.0013523046, 0.0063806702, 0.009216173, -0.016073106, 0.009714515, -0.013549856, -0.010881518, 0.005677314, -0.02124577, 0.029673427, -0.023150824, 0.005841325, 0.025610995, -0.023743788, 0.0011409824, -0.00568993, -0.053795703, -0.03532551, -0.023466231, -0.0122503815, -0.0004711382, 0.031061215, 0.017826766, -0.0066361492, -0.008875534, 0.004904568, -0.00946219, 0.0009564697, -0.010465182, 0.02535867, 0.0056205406, 0.0047594816, 0.004078204, -0.011796197, 0.038000155, 0.0046080863, 0.0073426594, 0.02798285, 0.009746055, 0.011342011, -0.001569935, -0.015543224, 0.045569908, 0.010048846, 0.0040561254, -0.01912624, -0.014483458, 0.0426934, 0.029320173, -0.00959466, 0.021119608, -0.0050654258, 0.026796922, 0.023327451, -0.0003221087, -0.013499391, 0.032802258, 0.022545245, 0.010919367, -0.03633481, 5.9532944e-05, -1.0453995e-05, -0.015126888, -0.020577108, 0.01609834, 0.0017915079, -0.034619, 0.0284875, -0.0073742, 0.00079955504, -0.0012805497, -0.0031114835, 0.003936271, -0.015404445, 0.019365948, -0.02118269, -0.021889199, -0.019567808, 0.0133479955, -0.0070020203, 0.008061786, -0.0041475934, -0.048572574, 0.0053902944, 0.01050303, -0.03090982, -0.026292272, 0.015694618, 0.0066550737, -0.038075853, -0.009449573, 0.0109004425, -0.03302935, 0.005734087, 0.0066992305, -0.04100282, 0.02562361, 0.011720499, 0.023112975, -0.0113735525, 0.0056047705, 0.00031974315, 0.011695267, -0.052988265, 0.016779616, 0.025989482, -0.009525271, 0.0073678917, -0.0076454496, 0.04700816, 0.007613909, -0.0055953083, 0.02148548, -0.02098083, 0.0061693476, -0.031162145, 0.0045639295, 0.04448491, -0.020728504, 0.00313987, -0.0043116044, -0.027806222, 0.025308203, -0.0027077633, 0.006623533, 0.018974844, 0.000684826, 0.02259571, 0.034164812, 0.016981477, -0.041835494, -0.028462267, -0.01916409, -0.028134244, 0.012477474, -0.013827413, 0.0072985026, -0.009310795, -0.020286934, -0.0029033152, 0.03482086, 0.00040115742, 0.009771288, -0.011619569, -0.0005452587, 0.012666718, 0.055662908, -0.02879029, -0.02400873, 0.008257338, -0.007109259, 0.024954949, -0.039816894, -0.005718317, -0.041305613, -0.036486205, -0.02687262, -0.0013783256, -0.017095024, -0.015139503, 0.00747513, -0.014420377, -0.0213467, 0.022343384, 0.033988185, -0.0034316208, -0.0022614633, 0.029446336, -0.03199482, 0.007096642, 0.0086358255, 0.012035905, 0.0068569337, 0.006159886, 0.02768006, -0.00027322074, -0.0012813382, -0.04751281, -0.0067812363, 0.0026699146, 0.0012419124, 0.030329473, -0.021207921, 0.014104971, -0.031364005, 0.015909094, 0.0062450455, -0.006579376, 0.0056173867, 0.00627974, -0.037041318, 0.0064059026, -0.01114646, 0.0063648997, -0.0336097, 0.0021652645, -0.028865987, 0.025989482, 0.016211886, -0.021397166, 0.029824823, -0.030329473, -0.0006607763, 0.033306908, -0.009531579, 0.0305061, -0.035375975, 0.024803553, -0.017826766, -0.0013018396, -0.022961581, -0.034391906, -0.0025516371, 0.009209865, -0.021119608, 0.0066361492, -0.0028370798, -0.01333538, 0.015278283, 0.007992396, 0.029723892, 0.0029616654, 0.013764332, 0.015946943, -0.0121873, 0.019101007, -0.028411802, 0.015467526, -0.030707961, 0.02899215, -0.022822801, -0.016640838, 0.016237117, -0.028083779, -0.04869874, 0.048774436, -0.012710875, 0.026721224, 0.030632263, 0.054199424, 0.018936995, -0.00777792, -0.045418512, -0.03199482, 0.023150824, -0.02185135, -0.008093326, -0.012805497, -0.039211314, 0.017864615, -0.023024661, -0.010282246, -0.02568669, 0.008251029, 0.016918395, -0.006623533, -0.009973148, -0.00022354424, -0.02148548, 0.002646259, 0.0073489673, 0.023049895, 0.010477798, 0.053089194, -0.021170072, -0.01807909, 0.023743788, -0.03141447, -0.00041278804, -0.0004289526, 0.0067118467, -0.007960856, 0.005147431, -0.003680792, -0.02350408, -0.017158104, 0.002690416, 0.04039724, 0.036990855, -0.040927123, 0.0026304887, 0.017044557, -0.0064721378, -0.0020028302, -0.008358268, 0.013019973, 0.010717507, 0.00936126, -0.011499715, 0.0003368934, 0.025270356, 0.0026557213, 0.024525996, 0.012736107, -0.030077148, 0.0022346538, 0.018104324, -0.013562473, 0.03565353, 0.010395792, 0.007033561, -0.039943058, -0.002540598, -0.009884834, -0.0110455295, -0.012521631, 0.022784954, 0.0006580165, -0.00479733, 0.0043052966, 0.023756405, 0.00066274754, -0.027074479, 0.009853294, 0.040220615, 0.008755679, 0.022065826, -0.0049550333, -0.034543302, 0.018142171, -0.0059075607, 0.006737079, 0.024525996, -0.013890495, -0.0120674465, 0.007815769, 0.00797978, 0.004138131, -0.03522458, -0.029976217, -0.0030105533, 0.03744504, -0.034417138, -0.010553496, -0.003232915, 0.025144193, -0.014773632, -0.05737872, 0.0084528895, 0.010944599, 0.015303515, 0.037192713, -0.004049817, 0.026115645, 0.021397166, 0.019782284, -0.005144277, -0.016350664, -0.0009288716, -0.021725187, -0.015883863, 0.030682728, -0.003841649, 0.010181316, 0.012868579, 0.030354705, 0.0012103717, -0.024097044, -0.028109012, -0.011638493, 0.009323411, 0.012130528, 0.015480143, 0.008301495, -0.0009454305, -0.0163002, 0.015530608, -0.0069452473, 0.026039947, -0.0060211066, -0.022444313, 0.038202014, -0.03149017, -0.008509663, -0.0013854223, -0.013398461, 0.0027487662, -0.025939016, 0.0067433873, 0.041658867, 0.0057403953, 0.0005357965, -0.016022641, -0.009910067, 0.012792881, 0.005932793, 0.008421349, 0.04723525, 0.0057246247, 0.024740472, -0.0027219567, 0.0067686196, 0.0044188425, 0.001316033, -0.037924457, 0.008219489, 0.010080386, 0.02255786, -0.034417138, -0.035073183, -0.021624258, -0.022015361, 0.0077526877, -0.0073426594, 0.026090411, -0.007468822, -0.030354705, 0.016073106, -0.025800237, 0.015089039, -0.0153918285, 0.03323121, -0.019239785, 0.01333538, -0.02909308, -0.00033511923, -0.008604284, -0.027957616, -0.00228985, 0.013713867, 0.020425715, 0.03310505, 0.011588029, 0.0066550737, -0.026241807, 0.005383986, -0.023478847, -0.014079738, 0.008301495, -0.01155018, 0.022507396, -0.009323411, 0.008320419, -0.0071029505, 0.011348319, -0.0253839, 0.009424341, 0.019101007, -0.021561177, -0.011770965, -0.011133843, 0.011878203, -0.012313463, -0.019403797, -0.018205253, 0.04440921, 0.007285886, -0.0026604524, 0.021535944, -0.008061786, 0.023529312, 0.0017710065, -0.012174685, 0.03898422, 0.022166757, -0.010925675, -0.0062198127, -0.03426574, -0.032070514, -0.003942579, -0.011272622, -0.013411077, -0.0029112005, -0.045670837, -0.0052105123, 0.03694039, 0.014420377, 0.02586332, 0.028159477, -0.03222191, 0.0620215, -0.009317103, 0.020148156, 0.004387302, 0.03202005, -0.020652806, 0.00747513, 0.038580503, 0.04511572, 0.0005058329, 0.004652243, 0.02023647, 0.014786249, 0.011808813, 0.023743788, -0.0239078, -0.011316779, -0.022961581, -0.05551151, 0.009544196, 0.019807518, 0.022974197, 0.019706588, -0.0018782447, -0.01918932, -0.0024081273, -0.009884834, 0.034593765, 0.0011520216, -0.02199013, -0.023377918, 0.027654827, 0.009979457, 0.010622885, -0.017221186, -0.03214621, -0.02521989, -0.008459197, -0.0035010101, -0.0019570962, -0.0055385353, 0.031338774, -0.0058192466, -0.034492835, -0.021977512, -0.006718155, 0.0049581877, 0.012363928, 0.022633558, 0.012244074, -0.03149017, -0.020148156, 0.00053461373, -0.021460246, 0.021573793, -0.025270356, 0.026317503, -0.043021422, 0.017511358, -0.011449249, 0.014609621, -0.009241405, -0.018369265, 0.052483615, 0.0028701976], "content_hash": "7df61a5f96fe8df2c86624bb74e00b8a"}, {"content": "- Another might spread attention broadly across the sentence GPT-2 has **12 attention heads per layer** (144 heads total across 12 layers). The dashboard categorizes these heads to help you understand their roles. ## What You See in the Dashboard In **Stage 3 (Attention)** of the pipeline, you can see:\n\n- **Head Categories**: Heads are grouped by pattern type -- Previous-Token, First/Positional, Bag-of-Words, Syntactic, and Other. Click each category to see which specific heads (like L0-H3) belong to it.\n- **BertViz Visualization**: An interactive attention map. Lines connect tokens on the left to the tokens they attend to on the right. Thicker, darker lines mean stronger attention. You can click on specific heads to focus on them.\n\n**Want more technical detail?** Ask the chatbot about Q/K/V dot products, softmax normalization, or how attention scores are computed mathematically.", "source_file": "attention_mechanism.md", "chunk_index": 1, "embedding": [-0.0076232045, 0.018918902, 0.033664517, 0.025512695, -0.020894257, -0.042261485, -0.0077692694, 0.0412599, 0.00879868, 0.013931823, 0.021200297, -0.008638704, -0.0023683405, -0.014070933, -0.011893868, -0.019252764, -0.006266886, 0.011615649, 0.0060164887, 0.0351669, -0.034415707, 0.00632253, 0.027390674, 0.06972171, 0.025206653, 0.035834625, -0.001527597, 0.02821142, 0.0057104477, 0.01585849, 0.005654804, -0.02325912, -0.0261526, -0.011031388, 0.0022083644, -0.0012685054, -0.011726936, 0.067217745, -0.032356884, 0.037003145, -0.011100943, -0.010363663, -0.029685982, 0.025554428, -0.034666106, -0.00036320637, 0.02049084, -0.019809203, -0.06326703, 0.07228133, -0.025540518, 0.053390253, -0.03661364, 0.027001167, 0.030993612, 0.009285564, -0.031911734, 0.021450695, 0.033358473, -0.017138299, 0.008290931, -0.0034794782, 0.01918321, 0.048159733, -0.014787347, -0.037281364, -0.086247936, -0.0026987256, 0.0050044665, 0.0077553582, 0.05144272, 0.054558773, -0.0056339377, 0.004576705, 0.03733701, 0.007991845, 0.014077888, 0.009633337, -0.0029247787, -0.039061967, 0.049940336, -0.027418496, -0.0070771994, -0.039729692, -0.032329064, -0.07361678, -0.029546872, -0.05714621, -0.0511645, -0.043875158, -0.030186776, 0.020337818, -0.017180031, 0.0009850697, 0.0070250332, 0.011483495, 0.05122014, -0.022813968, -0.016498394, -0.010780992, 0.03282986, 0.015635915, -0.045182787, -0.009382941, 0.017806025, -0.040397417, 0.025067544, 0.004110688, -0.019322319, -0.05216609, -0.08057226, 0.024385907, -0.005800869, 0.027599338, 0.068553194, 0.0034307898, 0.004747114, 0.0032499472, 0.020449106, -0.010168909, -0.012269464, -0.012408573, 0.034638282, 0.015538539, -0.01198429, 0.008367441, -0.0031716982, -0.029546872, 0.006840713, -0.020184798, -0.007456273, 0.017124388, 0.034860857, -0.01221382, 0.0128189465, -0.011733892, -0.017680826, 0.027557606, -0.040675636, -0.025651805, -0.0064477287, -0.019572716, 0.007588427, 0.035723336, -0.034415707, 0.018807614, -0.055449072, 0.059038103, -0.06454684, 0.038978502, -0.023926847, 0.006604227, -0.06410169, -0.031049255, -0.036557995, -0.015886312, -0.0070076445, 0.040759105, -0.018223353, -0.032941144, 0.028016668, -0.024232887, 0.027947113, 0.006520761, -0.028600927, 0.044375952, -0.027446318, 0.0003662494, -0.036585815, -0.06237673, -0.022285353, -0.027710626, 0.011622604, -0.029546872, 0.019864846, 0.008986478, -0.059538893, 0.021854114, -0.0050601107, -0.04242842, -0.0053000744, -0.013187587, 0.015496806, 0.005905201, -0.0735055, -0.011928646, 0.012763303, 0.018765882, -0.020003956, 0.024761504, -0.0033455852, 0.016053244, 0.06922092, 0.03650235, -0.016707059, -0.058370374, 0.050663706, 0.017639093, 0.0033299353, -0.0016962673, 0.023078278, 0.0231061, 0.0473529, -0.03853335, 0.0026361262, 0.003456873, 0.0019701393, 0.003025633, -0.038338598, -0.031299654, 0.028378353, -0.050051622, 0.025248386, 0.07183618, -0.078235224, 0.006159076, -0.06465813, 0.034443527, -0.0036203265, 0.032440353, 0.02862875, 0.0046879924, 0.05887117, 0.029825091, -0.06465813, 0.04838231, 0.056645416, -0.008513506, 0.015649827, -0.009076899, 0.03157787, 0.003947234, 0.011379163, -0.012547683, -0.023175655, -0.05714621, 0.027432406, -0.028183598, 0.018362463, -0.045878336, -0.00952205, 0.013208454, 0.022813968, 0.021325497, 0.0430405, 0.009821136, -0.0018831958, -0.03836642, 0.04593398, 0.013403207, 0.05480917, -0.01801469, -0.049495183, 0.004336741, -0.03180045, 0.010787947, -0.014084844, 0.030492818, 0.0012702442, -0.016651416, 0.021116832, 0.021311587, -0.05032984, -0.008423084, -0.0005042722, -0.054113623, -0.020908168, 0.014801258, -0.06571536, -0.047185965, -0.052972924, -8.335706e-05, 0.029964201, 0.010989656, -0.046045266, -0.00029930292, -0.0018866735, -0.01761127, -0.02164545, -0.0051261876, 0.004910568, 0.011838225, 0.04807627, 0.033914912, -0.0006068655, -0.013006745, -0.004162854, -0.011532183, -0.0006112127, -0.009695937, 0.041482475, -0.008680437, 0.067774184, -0.0062738415, -0.017847758, -0.0034968667, 0.030659748, 0.033024613, -0.0020727327, -0.00916732, 0.036418885, -0.05330679, 0.014286553, 0.007588427, 0.01903019, 0.034554817, 0.0231061, 0.018278997, -0.012561594, -0.051081035, -0.0120468885, -0.038004734, 0.033692338, -0.0038081245, -0.031160543, -0.039368007, 0.015566361, 0.033553228, -0.0004477589, -0.009862868, -0.016554039, -0.032746393, 0.012088621, 0.03196738, 0.009779403, -0.012109488, -0.022410551, 0.0349165, 0.004357607, -0.028155776, 0.06571536, 0.0136675155, -0.036251955, 0.018793702, -0.030576283, -0.02064386, -0.00074640976, 0.044069912, -0.03780998, 0.022076689, -0.060039688, -0.01918321, -0.04317961, -0.0039646225, -0.05130361, 0.059872758, 0.006990256, -0.016067155, 0.017527806, -0.017249586, -0.024399819, -0.024691949, 0.014940368, -0.02645864, 0.0015719382, 0.023926847, 0.0019562284, -0.002413551, 0.008443951, 0.013243231, -0.03068757, -0.015218587, 0.033497583, 0.0060582217, 0.014411751, -0.009682026, -0.024469374, -0.0270429, -0.014481306, -0.0054913503, 0.017666915, -0.010391485, -0.0025717882, -0.040091377, -0.04827102, -0.006426862, -0.0064929393, 0.024052044, -0.051804405, 0.029519051, -0.013876179, -0.0022883525, -0.0041558985, -0.0115113165, 0.032162134, 0.033330653, 0.006600749, 0.0083535295, -0.069833, 0.00040972116, -0.0045141056, -0.016762704, -0.013848358, 0.013041522, -0.0155107165, 0.022424463, -0.012603327, 0.030659748, 0.0012563333, 0.055755116, 0.015302053, -0.011692159, -0.023064366, -0.03152223, 0.041204255, -0.0033977511, -0.027084634, 0.004465417, 0.008423084, -0.036251955, 0.049050033, 0.0025822213, 0.010794902, -0.010419306, -0.027752358, 0.035278186, 0.015315964, 0.0027787136, 0.00021725002, 0.044598527, -0.00094507565, -0.04031395, -0.018237265, 0.0068859234, -0.032607283, 0.0008224853, 0.055643827, -0.028740037, 0.015371608, -0.022271441, -0.008555238, 0.0631001, 0.0016962673, -0.016707059, -0.0027700192, -0.022410551, 0.039173253, 0.04509932, -0.007915335, -0.0045036725, 0.022480106, -0.0319952, -0.021158565, -0.018918902, -0.048799638, -0.009271652, -0.02122812, -0.07066766, 0.033052433, -0.0520548, -0.0069972114, 0.028016668, -0.005654804, 0.04593398, -0.058370374, -0.028879147, 0.07912552, 0.0074214954, 0.038199488, -0.025623983, 0.015093388, 0.01903019, -0.030492818, 0.020323908, -0.044654172, 0.012568549, -0.0071363207, -0.03171698, 0.034860857, 0.008179642, -0.02239664, -0.0038602906, 0.018918902, -0.033720158, 0.0024448507, -0.017430428, 0.017180031, -0.013417118, -0.0034447007, -0.050190732, 0.03241253, -0.008986478, 0.0053661517, 0.025763093, 0.020421285, 0.06415733, -0.01628973, -0.007504961, 0.023231298, -0.006040833, -0.038644638, -0.018668504, 0.04551665, -0.016247997, 0.00879868, 0.0058078244, -0.035667695, -0.06298881, -0.027752358, -0.024052044, 0.021158565, -0.009202098, -0.0033403686, 0.018585037, 0.02559616, 0.007852735, 0.041232076, 0.021395052, -0.007240653, 0.024886701, -0.045154966, -0.019252764, 0.0021840201, 0.02051866, -0.0200596, -0.002401379, 0.045294076, 0.014842991, -0.0034099233, -0.012443351, -0.013862268, -0.010384529, -0.0022500972, 0.0039298455, -0.018807614, -0.02561007, -0.019544894, -0.015622005, -0.03024242, -0.022382729, 0.021450695, -0.05614462, 0.0055643828, -0.009869824, -0.07133539, 0.00070119917, 0.0030134611, -0.033553228, -0.025053633, 0.02296699, -0.007400629, 0.013027611, 0.026792502, 0.015900224, 0.0051922644, 0.0030204165, 0.023050455, -0.021325497, 0.004093299, 0.0631001, 0.0150516555, -0.021909757, 0.029407762, 0.0053244187, -0.051720936, -0.01859895, -0.027001167, -0.007824914, 0.003828991, -0.021673271, 0.05583858, -0.029825091, 0.015788935, -0.047603294, -0.018932812, 0.04070346, 0.013180631, 0.008541328, 0.025512695, -0.034972146, -0.0209499, -0.023926847, 0.018807614, -0.04782587, 0.008304841, -0.008889101, -0.014258731, 0.022076689, 0.00798489, 0.0115878275, -0.027223742, -0.032329064, -0.014550861, 0.0005912157, -0.008847369, -0.007852735, 0.026486462, -0.02367645, -0.04131554, 0.016595772, 0.03271857, -0.017472161, -0.013591005, 0.034137487, -0.027696716, -0.028030578, 0.014411751, -0.014564772, -0.025554428, 0.017082656, 0.0016423623, 0.0042741415, -0.029101722, -0.013319741, -0.01585849, 0.015149032, -0.027668893, -0.008332663, 0.0038602906, 0.01641493, -0.018515484, -0.018779792, -0.043958623, -0.026096955, -0.047881514, 0.020003956, -0.041204255, 0.001193734, 0.0069485228, 0.012394663, 0.014328285, -0.009668115, -0.003505561, -0.0050044665, 0.011866046, 0.030325886, 0.022313174, -0.032356884, -0.009341207, -0.0025665716, -0.037170075, 0.013750981, 0.0432909, -0.0015658521, -0.023523428, 0.004055044, -0.0091255875, 0.010655793, 0.048159733, -0.028141866, -0.0013024133, -0.01061406, -0.018000778, -0.03547294, 0.018835435, 0.0015823713, -0.01759736, -0.009779403, -0.017346963, -0.013848358, -0.01527423, -0.044181198, 0.045989625, 0.0019423174, 0.0041802423, -0.039145432, -0.038032558, -0.015107299, -0.0023196521, 0.022563571, -0.043207433, -0.0012615499, 0.0033281965, -0.045433186, -0.019322319, 0.022521839, 0.009035166, 0.011288742, 0.010495817, 0.010801858, -0.07689977, 0.050691526, 0.03536165, 0.022327086, 0.043068323, 0.004187198, -0.022507928, -0.05631155, -0.024038134, 0.027056811, -0.013486672, 0.024344174, -0.047603294, 0.0004864488, 0.03082668, 0.005202698, 0.02645864, -0.023147833, 0.025679626, 0.0046393042, 0.0032238644, -0.015009922, 0.033998378, -0.013500583, -0.017166121, 0.006259931, -0.01830682, 0.06326703, -0.012881546, -0.01773647, -0.020143066, -0.008673482, -0.015580271, -0.007908379, -0.0050149, -0.005418318, 0.026416907, 0.07261519, 0.010203687, -0.008624793, -0.03764305, 0.008423084, 0.0057800026, 0.0179034, -0.008749992, 0.03166134, -0.022507928, -0.0360572, 0.030131133, -0.019600539, -0.017513895, 0.0371979, -0.006280797, 0.00792229, 0.017138299, -0.02284179, -0.020838613, -0.046295665, -0.013938779, 0.017708648, 0.010148043, 0.039228898, -0.0029195622, -0.051943514, -0.009723759, 0.032217775, -0.040341776, 0.0010494078, -0.029463407, -0.041232076, 0.0340262, -0.022118421, -0.0034168789, 0.011427851, 0.019976133, 0.009320341, 0.0039576674, 0.0033925346, -0.036140665, -0.057368785, 0.0011815618, 0.012985879, 0.00038646377, 9.4496696e-05, -0.0030604105, 0.0071571874, 0.00915341, -0.025039723, 0.055616006, 0.026486462, 0.01309021, -0.003317763, -0.018849347, 0.038449883, 0.01801469, -0.04554447, -0.0020605605, 0.022104511, 0.0159976, 0.0100437105, 0.033942733, -0.013069344, 0.019141477, 0.006082566, 0.04201109, -0.045627937, 0.039201077, -0.014856902, 0.015594183, 0.0034777392, -0.018946724, -0.0046879924, -0.001133743, -0.035445116, 0.00290739, -0.0067711584, -0.022883523, -0.009925468, -0.0040654773, -0.021144655, -0.019906579, -0.019572716, 0.0012685054, -0.032217775, -0.010481906, -0.00040819964, 0.009334252, -0.10104919, -0.046573885, 0.0091255875, -0.030186776, -0.02092208, -0.009062989, 0.055643827, 0.011441762, 0.032301243, -0.020101333, 0.025832647, -0.007456273, -0.008339618, -0.008110088, 0.025234476, -0.050107267, 0.0057382695, 0.01832073, 0.028823502, -0.017472161, -0.03282986, 0.001287633, -0.012366841, 0.011344385, -0.039813157, 0.010523639, -0.0059191124, -0.0022605306, 0.023732092, 0.02239664, -0.019002367, 0.018487662, 0.00020018737, 0.032078665, 0.008416129, 0.023175655, -0.017708648, 0.009932423, 0.047853693, 0.0034064455, 0.0050879326, -0.031494405, -0.00632253, -0.02195149, 0.006249497, 0.009779403, -0.01221382, -0.04098168, 0.011810402, 0.0311049, -0.005345285, -0.0001862764, 0.0016554039, 0.025679626, 0.036001556, 0.012185998, -0.008256153, -0.0100298, 0.037837803, 0.00289174, -0.012672882, 0.003881157, 0.024552839, -0.010621016, 0.019141477, -0.007831869, -0.002321391, 0.04607309, -0.009529006, -0.0063712182, -0.03522254, -0.0063260077, 0.007574516, -0.005028811, 0.0031386595, -0.008270063, 0.012881546, 0.01773647, 0.021603717, -0.0070215557, 0.040953856, -0.024956256, -0.04479328, 0.021339407, 0.01287459, 0.014008334, 0.023467785, -0.033052433, 0.016985279, 0.019169299, -4.529212e-05, -0.01992049, -0.008673482, 0.024302442, -0.021715004, 0.0036307597, 0.020309998, 0.021311587, 0.06321139, 0.0018988457, 0.04854924, -0.0005916504, -0.022605306, -0.035584226, -0.056923635, -0.005439184, -0.00071989204, -0.009118632, -0.0009415979, -0.03622413, 0.008687393, 0.002253575, 0.024024222, 0.014523039, -0.005613071, 0.033720158, 0.077289276, 0.008242242, 0.0051853093, -0.012408573, 0.014091799, -0.0054739616, -0.027307209, 0.009452495, -0.0047088587, -0.0029960724, -0.023954667, 0.029407762, 0.0073936735, -0.019670093, -0.0050844545, 0.011073122, -0.007664937, 0.00059338927, 0.0007555388, 0.0051400987, 0.01845984, 0.005439184, -0.021742826, 0.033775803, -0.024942346, -0.011525228, 0.036724925, 0.020407373, -0.0136675155, -0.008492639, -0.002933473, -0.01830682, -0.01287459, 0.010071533, 0.031466585, 0.007477139, 0.016025422, 0.02630562, -0.0011128766, 0.028475728, 0.0036029378, -0.028044488, -0.018682415, 0.028545283, -0.0035281663, 0.030353708, 0.047853693, 0.00036494527, 0.010822725, 0.00719892, -0.025220565, 0.005776525, -0.029936379, 0.038143843, -0.054976102, 0.021826291, 0.04070346, 0.048771814, 0.002293569, 0.025554428, -0.006788547, 0.005856513, 0.034137487, 0.015093388, 0.002519622, 0.018084245, -0.02007351, 0.012039933, 0.048771814, 0.015079477, -0.007477139, 0.02021262, -0.01438393, -0.014787347, 0.01024542, 0.009515095, -0.018348552, 0.023732092, -0.014550861, -0.035389476, 0.03792127, 0.01127483, -0.020254353, 0.01761127, -0.0034690448, 0.01293719, 0.007191965, 0.00027126365, -0.0049175234, 0.024872791, -0.009563783, -0.0241216, 0.0077136257, -0.015455073, -0.008075311, 0.031605694, -0.021200297, 0.010913146, -0.015955867, 0.020699503, 0.0471025, 0.0048375353, 0.014147443, -0.0010433218, -0.046712995, -0.009292519, -0.03575116, 2.1953228e-05, 0.019364052, 0.02106119, 0.014982101, -0.025067544, -0.020421285, 0.011226142, -0.0068372353, 0.003053455, -0.0007225003, 0.012526817, 0.008513506, -0.019224942, 0.0060373554, 0.0002851746, 0.015162943, 0.008165732, 0.017374786, 0.02893479, 0.018432017, 0.0083535295, 0.00013084993, -0.022660948, 0.0115113165, -0.0024587617, -0.008332663, -0.02164545, 0.0008424823, 0.014912546, 0.02559616, -0.010154999, 0.00065338024, -0.0130484775, 0.034693927, 0.007880557, 0.015455073, 0.0045106276, 0.014286553, 0.048465773, -0.00018040772, -0.022869613, 0.007226742, -0.0013850096, -0.017180031, -0.0013824013, 0.03241253, 0.016623594, -0.043819513, 0.035862446, -0.0049140453, 0.00057165336, 0.000979853, 0.009848958, 0.035834625, -0.02876786, 0.021603717, -0.007331074, 0.0011772147, -0.0064442507, -0.0030099833, -0.011685204, 0.006816369, -0.0029021734, -0.043346543, -0.0072336975, 0.021200297, -0.02776627, -0.012325108, -0.009007345, 0.013333652, -0.012290331, -0.0070458995, 0.017541716, -0.009786358, 0.004319352, 0.022591393, -0.05060806, 0.018432017, 0.011615649, 0.027154189, -0.019155387, 0.0054044067, 0.011330474, 0.01775038, -0.044292487, 0.0029978112, 0.0058704237, 0.017027011, 0.014926457, -0.016247997, 0.033080254, 0.012700704, -0.004823624, 0.018793702, -0.020296086, 0.01322932, -0.016540127, 0.0056513264, 0.044542883, -0.029936379, -0.0008733473, 0.013104121, -0.026862057, 0.00509141, -0.008840413, 0.010711437, -0.011747803, 0.03505561, 0.015218587, 0.00021257681, 0.014801258, -0.06655002, -0.0239964, -0.003561205, -0.021395052, 0.0045627938, -0.020296086, 0.014161354, 0.007762314, -0.0054878723, -0.0048688347, 0.044542883, 0.0040724324, 0.01773647, -0.014884723, 0.0063433964, -0.0028743516, 0.05208262, -0.018292908, -0.026486462, 0.019739648, -0.007824914, 0.03722572, 0.009904602, -0.016623594, -0.03327501, -0.011191364, -0.006496417, 0.00045949628, 0.0052166088, -0.018098155, -0.0016693148, -0.010099354, -0.023147833, 0.0064616394, 0.04028613, -0.011469584, -0.0048966566, 0.016512306, -0.03736483, 0.019739648, 0.028461818, 0.016957456, 0.020017866, 0.004385429, 0.012283375, -0.0011285264, -0.0071641426, -0.048604883, 6.950044e-05, 0.017541716, -0.0046984255, 0.032495994, -0.007191965, 0.029797269, -0.010342796, 0.0023822514, 0.005150532, 0.021297675, 0.013716203, 0.012332063, -0.01583067, -0.005574816, 0.011080077, 0.014272641, -0.015135121, -0.028280975, -0.033052433, 0.019224942, 0.008040533, -0.0026969868, 0.023954667, -0.032440353, -0.017945135, 0.018515484, 0.00012519861, 0.028002756, -0.038171664, 0.0120468885, -0.0073936735, -0.0034064455, -0.0190441, -0.043791693, -0.004319352, -0.015343785, -0.012958056, 0.02776627, 0.011504361, -0.018098155, 0.0019040622, -0.011942556, 0.034054022, 0.0049662115, 0.00071597955, 0.0005699145, -0.009132544, 0.029491229, -0.019753559, 0.007070244, -0.03152223, -0.007657982, -0.005762614, -0.009647248, 0.024719771, -0.029324297, -0.033191543, 0.024358086, -0.003081277, 0.015733292, 0.010175865, 0.05786958, 0.018835435, -0.009382941, -0.008158776, -0.02442764, 0.011886912, -0.012380752, -0.0027039421, -0.031633515, -0.018418107, 0.01527423, -0.018390285, -0.016108887, -0.021575894, 0.016762704, 0.018376375, -0.010516684, 0.01453695, 0.009828091, -0.0040446105, 0.003161265, 0.008562194, 0.012352929, -0.0060616992, 0.046712995, -0.033692338, -0.0084300395, 0.029908556, -0.012561594, 0.020727325, 0.002613521, 0.0063329632, 0.0147456145, 0.0064477287, -0.01759736, -0.033219364, -0.046852104, -0.011212232, 0.0418998, 0.028044488, -0.01344494, -0.0058495575, 0.024399819, 0.003933323, -0.018710237, -0.03764305, 0.030325886, 0.022633126, 0.020546483, 0.011330474, -0.019322319, 0.021589804, -0.0125129055, 0.022327086, 0.012102532, -0.0128189465, -0.00054730923, -0.0030169387, -0.017499983, 0.0006864188, 0.02149243, 2.4738138e-05, -0.052110445, -0.006924179, -0.004823624, -0.029157365, -0.010940967, 0.023189565, -0.009570738, -0.02876786, -0.009396851, 0.024997989, 0.012575505, 0.0013250186, 0.0024744114, 0.013625782, 0.01583067, 0.033831447, -0.027488051, -0.026556017, 0.016011512, 0.023746002, -0.0015041222, 0.0041385097, 0.02108901, -0.010036755, 0.019419696, -0.030604105, 0.02051866, -0.031299654, -0.045878336, 0.010008933, 0.011956467, -0.0371979, -0.014098755, -0.0018292909, 0.023231298, 0.0061799427, -0.043430008, 0.0018153798, -0.013375385, 0.00090334273, 0.051720936, -0.0030638883, 0.04390298, 0.031077078, 0.002521361, -0.014606505, -0.013709248, -0.026333442, -0.0023752959, -0.032440353, 0.023662537, -0.009320341, -0.005745225, -0.0006825063, 0.0069276565, -0.0075258277, -0.016860079, -0.049161322, 0.009118632, 0.004041133, -0.0100506665, 0.01512121, 0.0041524204, 0.01024542, -0.04713032, 0.029602516, 0.00079118565, 0.035278186, -0.008339618, -0.015232498, 0.02558225, -0.020532573, -0.020838613, 0.00270742, -0.02515101, 0.010447129, -0.05583858, 0.01527423, 0.017666915, 0.01816771, 0.017388696, -0.02776627, -0.004823624, -0.006458162, 0.0018623293, 0.014189176, 0.044737637, -0.0119982, 0.017054833, -0.0022187976, 0.015218587, -0.011219187, 0.005094888, -0.026695127, 0.023231298, 0.016011512, 0.012172087, -0.037837803, -0.053891048, -0.02921301, -0.0074075847, 0.0014693448, 0.027641071, 0.013945734, -0.009626382, -0.06527021, 0.011379163, 0.0038150798, 0.0006764203, 0.018000778, 0.026055222, -0.011170498, 0.008221376, -0.03182827, 0.010760125, -0.02106119, 0.013855313, -0.0064199064, 0.013723159, 0.011302653, 0.060874347, 0.0033751458, 0.032162134, -0.021130743, 0.008158776, 0.020449106, -0.017040921, 0.020546483, -0.024010312, 0.009633337, 0.0018484184, -0.0016527956, 0.010224553, 0.034972146, -0.023787737, -0.012728525, 0.014084844, -0.014342196, -0.016957456, -0.009264697, 0.0116643375, -0.007873601, -0.010878368, -0.030798858, 0.024024222, -0.028461818, -0.017430428, 0.013827492, -0.016804436, 0.01046104, 0.024552839, 0.0038881125, 0.019795291, -0.0028360963, 0.0036168487, 0.0052444306, -0.022118421, -0.01818162, -0.004941867, 0.0015562883, 0.0134379845, 0.010996612, -0.029101722, 0.0038359463, 0.022507928, 0.0011041822, 0.017541716, 0.03806038, -0.0051122764, 0.043708228, -0.013159765, 0.0008446559, 0.0147456145, 0.03561205, -0.017138299, 0.009264697, 0.042233665, 0.049161322, -0.0015189026, 0.010920101, -0.008826502, -0.00073032524, 0.022452284, 0.023746002, -0.01889108, -0.022132332, -0.012081666, -0.03619631, -0.009779403, 0.006548583, 0.0351669, -0.018557217, -0.029046077, -0.0027022033, 0.008756948, -0.022299264, 0.007518872, -0.026709037, -0.012554638, 0.0016484484, 0.012492039, 0.008652615, 0.00981418, -0.020866435, -0.037142254, -0.041621584, -0.017291319, -0.003347324, 0.001921451, -0.021548072, 0.011803447, -0.018501572, -0.034276597, -0.009758536, 0.010001978, 0.002079688, 0.0073728072, 0.03806038, -0.001971878, -0.005400929, -0.011455673, -0.030715393, -0.027362851, 0.033914912, -0.015775025, 0.025373586, -0.02312001, 0.006823324, -0.009480317, 0.016915724, 0.0034186176, 0.0015388996, 0.04245624, 0.0038116022], "content_hash": "bbf918757a7cbeef7accb9f249d13fc6"}, {"content": "# Attribution Panel Guide\n\n## What Is Token Attribution?\n\nToken attribution measures **which input tokens had the most influence on the model's prediction**. It answers the question: \"Which parts of my input mattered most for this output?\"\n\nFor example, if your prompt is \"The capital of France is\" and the model predicts \"Paris,\" attribution will show that \"France\" and \"capital\" had the highest influence on that prediction.\n\n## How to Use the Attribution Panel\n\nThe attribution panel is in the **Investigation Panel** at the bottom of the dashboard, under the \"Token Attribution\" tab.\n\n### Step-by-Step\n\n1. **Run an analysis first**: Click \"Analyze\" on a prompt so the model has made a prediction.\n2. **Choose an attribution method**:\n   - **Integrated Gradients**: More accurate but slower. Computes attribution by gradually \"building up\" the input and measuring how each token contributes along the way.\n   - **Simple Gradient**: Faster but less precise. Takes a single gradient measurement to estimate token importance.\n3. **Choose a target token (optional)**: By default, attribution is computed for the model's top prediction. You can select a different token from the top-5 predictions dropdown to see which input tokens would drive *that* alternative prediction.\n4. **Click \"Compute Attribution\"**.\n\n### Reading the Results\n\nThe results have two visualizations:\n\n- **Color-coded token chips**: Your input tokens are displayed as colored boxes. Darker blue = higher influence. Hover over any chip to see the exact attribution score.\n- **Horizontal bar chart**: Shows the same information as bars, with attribution scores labeled. Longer bars = more influential tokens.\n\n### What Attribution Scores Mean", "source_file": "attribution_panel_guide.md", "chunk_index": 0, "embedding": [-0.0053097797, 0.009309273, -0.023023605, 0.0014335158, 0.028551767, -0.021625966, -0.015024618, 0.04143001, 0.0075871823, 0.011917368, -0.005727824, 0.01407622, -0.011599155, -0.0595494, -0.014750082, -0.02181315, 0.004065008, 0.0052473852, 0.004592242, 0.06993186, -0.039982457, -0.009384147, 0.04073119, 0.037811123, 0.009415344, -0.014063741, -0.018930519, -0.020839794, -0.010345023, 0.036188863, -0.00033654083, -0.016572004, -0.03544013, -0.04544822, -0.02630556, -0.044724446, -0.025095105, 0.0030292575, 0.029225627, -0.0043333047, -0.020615174, 0.030373689, -0.033218883, 0.030223941, -0.04559797, 0.0046203197, 0.0051756315, -0.0662381, -0.038984142, 0.06174569, -0.026480265, 0.043177057, -0.006021078, -0.0072876886, -0.024795612, 0.051413145, -0.017620234, 0.015848227, -0.024196623, -0.052810784, -0.015885664, -0.017021244, 0.04118043, 0.036313653, 0.034092404, -0.03197099, -0.04302731, 0.011156153, -0.006869645, -0.019766606, 0.05725328, 0.026180772, 0.04949139, 0.004951011, -0.037386842, -0.030248899, -0.024408765, 0.04187925, -0.032644853, -0.017595276, 0.0048262216, 0.0015941819, -0.019716691, -0.013477232, -0.00014789468, -0.027503537, -0.040531527, -0.02354772, -0.01606037, -0.07008161, -0.027453622, -0.0033443503, -0.02687959, 0.031471834, 0.018755814, 0.005250505, -0.027852947, -0.019254971, 0.004398819, -0.0053690546, 0.036413483, -0.028077567, 0.013102865, -0.01811939, 0.014725124, -0.02306104, 0.009839627, -0.021214161, -0.01073187, -0.004005733, -0.028327145, 0.038709607, -0.071030006, 0.051762555, -0.008504383, -0.03162158, -0.042752776, 0.04869274, 0.037436757, -0.04856795, -0.026155813, -0.028801344, -0.0015824828, 0.033992574, 0.011124956, -0.0035689708, -0.012616186, -0.028926134, -0.00565295, -0.018606069, -0.032170653, 0.0058682114, -0.034117363, -0.042278577, 0.0031244094, -0.032495104, 0.017245866, 0.039533213, -0.057652604, 0.0120920725, -0.0074249567, -0.018893084, -0.010326305, -0.03104755, -0.06474063, -0.014737603, -0.0032944346, 0.03266981, -0.035490043, 0.028476892, -0.0035720905, -0.012840807, -0.018481279, -0.039083973, -0.036762893, -0.03162158, 0.012553792, 0.03381787, -0.008211128, -0.06469072, -0.012372848, 0.030972676, -0.016646877, -0.020041144, -0.05805193, 0.029999321, -0.03349342, -0.011761381, -0.034541648, -0.013439795, -0.027952777, -0.041504882, 0.0083795935, -0.039158847, 0.030149067, -0.040656317, -0.0863042, -0.011667789, 0.011218548, -0.018992914, -0.030398646, 0.024733217, -0.013951431, -0.0011472806, -0.017570317, 0.0019482712, -0.018718379, 0.018618546, -5.288917e-05, -0.017882291, 0.04484923, 0.0013461633, 0.04821854, 0.019092746, -0.04085598, -0.019230014, 0.047270145, 0.016572004, 0.024882963, -0.002110497, 0.0041118036, 0.010594602, 0.0410806, -0.016796624, -0.018705899, 0.016996287, -0.011698986, -0.040431697, -0.039158847, -0.019654296, -0.0006013279, -0.046770986, 0.034092404, -0.0057902182, -0.07397503, -0.0066263056, -0.023260705, 0.038160533, 0.039782792, 0.033742998, 0.023210788, 0.016547047, 0.021987855, -0.033418544, -0.017520402, 0.03509072, 0.031022592, -0.019454634, 0.028776387, -0.03566475, 0.025931193, 0.027253957, -0.011355816, -0.020864751, -0.004355143, -0.037486672, 0.005300421, -0.01581079, 0.03324384, -0.06024822, 0.032869473, 0.013302527, 0.05121348, 0.01234165, 0.009072174, -0.004648397, -0.007225294, -0.03521551, 0.029450249, 0.037237093, 0.04292748, 0.0025441395, -0.032245524, 0.039208762, 0.023385493, 0.019217534, -0.006264417, 0.016097805, -0.0065514324, -0.022462053, 0.03406745, -0.02341045, -0.062294763, -0.020016186, -0.017932206, 0.011661549, -0.034916013, 0.007843001, -0.06983203, -0.037686333, 4.808771e-05, -0.009090892, 0.015062055, 0.0042896285, -0.02259932, -0.008473186, -0.024733217, 0.0058806906, -0.035739623, -0.005459527, 0.0018609187, 0.0018905562, 0.051313315, 0.03544013, -0.040406737, -0.017707586, 0.038010787, 0.04582259, 0.0138516, 0.004511129, 0.010008093, -0.03588937, 0.047345016, -0.0045204884, -0.021401346, 0.016359862, 0.01281585, 0.015087013, -0.019804044, 0.01038246, 0.008747722, -0.04327689, 0.0093467105, -0.04652141, 0.032969303, -0.02815244, 0.024209103, 0.028252272, 0.01719595, -0.030498477, -0.040756147, 0.003300674, -0.008117537, 0.033193924, -0.022087686, 0.0620951, 0.07018144, -0.011661549, 0.041030686, -0.0034192237, 0.013227654, -0.043127142, 0.015648564, 0.011106238, -0.002941905, -0.007930352, -0.014475546, 0.028576724, -0.011742663, -0.0042303535, -0.00015257428, 0.039907582, -0.014013826, 0.059599314, -0.05495716, -0.004763827, -0.0060304375, 0.0011020445, -0.0024302695, 0.040556487, -0.025456995, -0.032495104, -0.0102077555, -0.03069814, 0.012547553, -0.0011402612, 0.007899156, 0.04270286, 0.015224281, -0.030648224, 0.0135271475, 0.019641818, -0.0044736923, -0.04489915, 0.025706572, 0.026155813, -0.0069382787, 0.0059462045, 0.017632712, 0.0030136588, -0.027303874, 0.028726472, 0.035864413, 0.036213823, -0.0083172, -0.0507892, -0.023260705, -0.010981448, 0.010170319, -0.027902862, -0.020727484, 0.013327485, -0.027902862, -0.0442752, -0.037411798, -0.0088350745, 0.006532714, 0.012266777, -0.015374028, -0.01776998, 0.045048896, -0.057452943, 0.014450588, -0.026105898, 0.065639116, -0.00027746096, 0.024096793, -0.0053472165, -0.019517029, 0.030174026, -0.03034873, -0.008847553, 0.006021078, 0.034416858, -0.0034223434, 0.038135577, -0.00704435, 0.039957497, 0.05216188, 0.03429207, -0.02920067, -0.004592242, -0.04257807, 0.003104131, 0.037237093, 0.059150074, -0.008254805, 0.0031119303, 0.030598309, 0.030049236, 0.059749063, -0.053359855, 0.04245328, -0.053160194, 0.025257332, 0.03034873, 0.05355952, 0.014987181, 0.030797971, 0.024995275, -0.0033880265, -0.02840202, 0.013090386, -0.044824276, -0.030074194, 0.005865092, 0.030149067, -0.017682627, 0.023859693, -0.03359325, 0.030947719, 0.027428662, 0.07043102, -0.021264078, -0.007412478, -0.012372848, 0.062344678, 0.01222934, 0.0043676216, 0.0025207417, 0.016796624, -0.021588529, -0.010014332, -0.050814155, -0.065639116, -0.020390553, -0.02042799, -0.05530657, 0.022312306, 0.019878918, -0.010794264, 0.05585564, -0.021513656, 0.064940296, -0.05565598, -0.048493076, 0.021638446, 0.05146306, 0.044924106, -0.005840134, 0.051038776, 0.020278243, -0.012978075, 0.030947719, -0.0033911462, -0.018269137, -0.018518716, -0.017507924, 0.014500503, -0.00093981856, -0.010713152, -0.010033051, -0.009527654, -0.018156826, -0.004161719, -0.038534902, 0.018843167, -0.021850586, 0.0042303535, -0.040656317, 0.019279929, 0.012285495, -0.02792782, 0.0042615505, 0.005412731, 0.055556145, 0.01685902, -0.018443841, -0.009945698, -0.027977735, 0.0056997463, -0.054507915, 0.017532881, -0.012672342, 0.020615174, -0.041704547, -0.006407925, -0.012297974, 0.015723437, -0.0184688, 0.034491733, -0.013027991, -0.046995606, 0.061945353, 0.0066263056, 0.005050842, 0.061945353, 0.027079254, -0.029674869, 0.05241146, -0.05425834, -0.025182458, -0.019978749, -0.015548733, -0.036263738, 0.01164907, -0.024870485, 0.030623266, 0.02815244, 0.0066512637, 0.017408092, 0.03349342, -0.0148374345, -0.02677976, 0.021650923, -0.009721078, -0.008916187, -0.00080645015, -0.040207077, -0.024546033, 0.0050820396, -0.060198303, 0.0019373521, 0.022324786, -0.04085598, 0.013651937, -0.007281449, 0.012429003, -0.04547318, 0.0034566605, 0.041604716, -0.01582327, -0.016921414, 0.011667789, 0.0043894597, -9.003345e-05, 0.002225927, -0.04961618, -0.0046140803, 0.018980436, -0.00013171109, -0.036213823, 0.0057371827, -0.037661377, -0.016409779, -0.0145504195, 0.006251938, -0.027753115, -0.019941311, -0.04038178, 0.024558512, 0.0137517685, 0.016334904, -0.023984483, -0.009845867, 0.03299426, 0.038035743, 0.020989541, 0.01811939, 0.005493844, 0.019554466, 0.039433382, -0.00038489664, 0.022287348, 0.039832707, 0.0026377314, -0.018930519, 0.0062301, -0.001442875, 0.015760874, -0.015760874, 0.009721078, -0.01534907, 0.011736423, -0.03104755, -0.014338277, -0.0009959737, 0.008616693, -0.008254805, 0.010756828, 0.0024177905, -0.015723437, 0.01050101, 0.033069134, -0.0140387835, 0.040631358, 0.013514669, -0.020739963, -0.022786506, 0.016671835, 0.02827723, -0.040306907, -0.028826302, 0.0019576303, 0.015161886, 0.024321413, -0.019878918, 0.031147381, -0.012622426, 0.006311213, -0.008629172, -0.06938279, -0.014862392, 0.011243505, -0.009596288, 0.022561885, -0.012978075, 0.031921074, 0.009427823, -0.0030838528, 0.005066441, -0.011218548, -0.011836254, -0.0017517282, 0.0003184074, 0.0456978, 0.018056996, -0.019304886, 0.023585156, 3.5145706e-05, -0.039283637, 0.016596962, 0.029749742, 0.021176726, 0.01420101, 0.02283642, -0.0049416516, 0.008410791, -0.0009585369, -0.0038934227, 0.033418544, 0.030523434, -0.011611634, -0.017270824, 0.0019139542, -0.000911741, 0.019354803, 0.022287348, -0.030623266, -0.03174637, -0.014438109, -0.023023605, 0.01685902, -0.009153287, -9.517613e-06, -0.0049572503, 0.03187116, 0.022674195, -0.040656317, -0.0060179587, -0.0343919, -0.011642831, -0.02595615, -0.043526467, -0.02805261, 0.018518716, -0.002458347, 0.030223941, 0.036987513, -0.012029678, -0.10632039, 0.033742998, -0.008267283, 0.038010787, -0.0110064065, 0.016260032, 0.017782459, -0.033568293, -0.028726472, 0.054208424, -0.007811803, 0.009602528, -0.029050922, -0.01084418, 0.013776726, -0.005734063, 0.023959523, 0.01431332, 0.006270657, -0.039333552, -0.008441988, -0.010139122, 0.003403625, -0.0077369297, -0.04996559, 0.0070006736, 0.0143257985, 0.05630488, 0.0029029085, -0.059050243, -0.011954804, -0.018705899, -0.037386842, -0.02827723, -0.0050758, 0.023447888, -0.017632712, 0.017058682, 0.010045529, -0.041979082, -0.03177133, 0.012123269, 0.01338988, 0.00041960363, -0.023934565, 0.03553996, -0.014238446, 0.036912642, -0.014862392, -0.065738946, -0.034791224, 0.0059243664, -0.04799392, 0.021126809, 0.037861038, -0.027753115, -0.014562898, 0.008797638, -0.008248565, 0.030872844, -0.005340977, -0.01153052, -0.019928833, -0.040681276, -0.009914501, 0.010107924, 0.0022586843, 0.01084418, -0.0017174112, -0.007818042, 0.014176052, 0.0030713738, 0.011486845, -0.033767954, 0.038609773, 0.039707918, -0.04175446, 0.02550691, -0.029275544, -0.044574697, -0.0021214162, 0.009321752, 0.024283975, -0.018206742, 0.015486338, 0.0048168628, 0.009402866, -0.06329308, -0.008217368, 0.023148393, -0.008061381, 0.024945358, -0.0021338952, 0.007605901, -0.034841143, -0.022362223, 0.0487177, 0.025806405, -0.011337098, 0.02920067, -0.020615174, -0.007200336, 0.02655514, -0.015735917, 0.043751087, -0.014737603, 0.036138948, -0.0054657664, -0.022299828, 0.007780606, 0.011137435, 0.0030963316, -0.022911295, -0.0028951091, 0.01858111, -0.0054408084, 0.00854182, -0.029749742, 0.008098818, 0.0006418844, -0.020964583, -0.008354636, 0.006657503, -0.031471834, -0.003968296, 0.017957164, -0.0080801, -0.0671865, -0.03219561, 0.04467453, -0.027328832, -0.010681954, -0.017857332, 0.037711293, 0.0005677908, -0.026230687, 0.0022134483, -0.030024279, -0.0044268966, 0.0011667788, -0.0042896285, -0.031097466, -0.0102077555, -0.011698986, 0.006323692, 0.004639038, -0.007468633, -0.02319831, -0.005702866, 0.0032226807, -0.0118050575, -0.04717031, -0.009140808, -0.044000667, 0.021488698, 0.024296455, 0.01304047, -0.013951431, 0.036712978, 0.036837768, -0.01708364, 0.016260032, 0.040806063, 0.0035128156, 0.010463573, -0.016197637, -0.039882623, -0.006545193, -0.040681276, 0.008666609, -0.024633385, 0.082810104, -0.05760269, -0.018493757, -0.015735917, -0.02412175, 0.023160873, -0.016834062, 0.026180772, -0.001108284, -0.0070318705, 0.030548394, 0.015461381, 0.030997634, -0.016759187, -0.011505563, 0.022674195, -0.0067323768, 0.022000333, 0.0038684648, 0.018094432, 0.005612394, 0.02181315, -0.0040182117, -0.0046359184, -0.044025626, -0.023373015, -0.009134568, 0.024383808, 0.0010310706, -0.022898816, -0.03336863, 0.0038809436, 0.026954465, -0.007936592, -0.05226171, 0.010413658, 0.05919999, -0.0071940967, 0.011149914, 0.05033996, -0.005344097, 0.03346846, 0.0075996616, -0.013789205, -0.0068010106, 0.027528495, 0.047245186, -0.008710286, 0.0106070805, 0.00075653446, -0.0143257985, -0.01257875, 0.017857332, 0.049241815, 0.021975376, -0.015386507, -0.0067198975, 0.019404719, -0.05111365, -0.018793251, -0.008722764, -0.019292409, -0.018967956, -0.03197099, -0.022312306, -0.0031821243, 0.017969644, 0.043301847, -0.02447116, 0.00058884895, -0.016210115, 0.010438615, 0.034441814, 0.02598111, 0.025182458, -0.012366609, 0.020777399, 0.00095619715, -0.004096205, 0.0025534988, 0.0023444768, 0.039982457, -0.017720064, 0.0044019385, 0.01453794, 0.00900354, 0.029450249, 0.016322426, -0.00842951, -0.018144349, -0.046172, 0.0077244504, 0.0064141643, -0.013215175, -0.05390893, 0.006208262, -0.016247552, 0.029450249, 0.03012411, 0.015436422, -0.056754123, -0.009240639, -0.029375374, -0.04567284, 0.016522089, 0.017520402, 0.024483638, 0.03034873, 0.00083608757, 0.010869138, -0.033193924, 0.020552779, 0.025207415, -0.018294094, -0.0033849068, 0.040256992, 0.0023553958, 0.028576724, 0.02366003, 0.0021900502, 0.043801006, -0.017345697, 0.013764247, 0.009047216, 0.005241146, 0.015598648, -0.014700166, -0.010307587, 0.016696794, 0.021401346, 0.009820909, 0.028327145, -0.00022793525, 0.0051725116, -0.005340977, 0.034192238, -0.02665497, 0.019417198, -0.0221376, 0.012915681, -0.0032164413, -0.03301922, 0.000346095, -0.0066450243, -0.038509943, -0.013477232, 0.025544347, 0.024957838, 0.019367281, 0.028027652, -0.037811123, 0.009090892, 0.012029678, 0.007406238, -0.014987181, 0.014176052, -0.034366943, 0.0044393754, 0.027029337, 0.017620234, 0.024770653, 0.02412175, -0.015561212, -0.0010131322, 0.025294768, -0.010987688, -0.0044799317, 0.014587856, -0.03212074, 0.030997634, 0.00077291304, -0.008111297, -0.040556487, -0.006451601, -0.017819896, -0.0043239454, -0.038335238, -0.03256998, 0.004395699, 0.027977735, 0.0049416516, -0.004813743, 0.05206205, -0.015448901, 0.009989374, 0.0042178747, -0.017969644, -0.017819896, -0.014762561, 0.0014257164, -0.020228326, 0.008997301, 0.0038466267, -0.014263404, 0.034691393, -0.017645191, 0.03578954, 0.03212074, -0.0036968796, 0.007818042, -0.03346846, -0.03336863, 0.012329171, 0.0063642487, -0.014151094, 0.0013937393, 0.007293928, 0.07018144, 0.030723099, -0.042178746, 0.011380774, -0.008286002, 0.05600539, 0.013801684, 0.0065826294, 0.0073438436, 0.0052723433, 0.027253957, 0.018955477, -0.0074935905, 0.008816356, -0.012716018, -0.023223268, 0.0045298473, 0.011929846, 0.011917368, -0.017220907, 0.017882291, 0.0021136168, 0.01627251, -0.0023522761, -0.008697806, 0.0015590849, -0.034940973, -0.008760201, -0.010401179, -0.015486338, 0.024907922, -0.031322084, 0.011611634, 0.011274703, 0.015498817, 0.02191298, 0.0060616345, -0.022474533, 0.0020824196, 0.0011020445, 0.015136928, 0.018306574, 0.013901516, 0.01211079, 0.005759021, 0.0029075881, 0.007200336, -0.005843254, -0.023797298, 0.042727817, -0.0203032, -0.018830689, -0.012865765, 0.015785832, 0.015461381, 0.025706572, 0.00055219216, 0.003971416, 0.0019420317, -0.03222057, -0.0009047216, -0.031322084, 0.037486672, -0.013602021, -0.020939626, -0.016983809, -0.058551088, -0.010900335, -0.014263404, 0.0082048895, -0.004002613, 0.0016908934, 0.025007753, 0.011050083, -0.0140887, 0.014475546, 0.01200472, -0.008048902, 0.010806744, -0.019729171, -0.016309947, 0.011025124, -0.0019108345, -0.042727817, 0.012940639, -0.016572004, -0.019928833, 0.0038372674, -0.06269409, 0.01131214, 0.0052099484, -0.0002942295, -0.016896456, 0.02907588, 0.012510116, 0.02191298, -0.008123776, 0.020353116, 0.031322084, 0.0671865, -0.02294873, 0.02087723, 0.005150674, 0.0046951934, 0.026704885, -0.02294873, 0.017645191, -0.015636085, 0.011343337, 0.0069258, 0.040232033, 0.010557165, -0.015274197, -0.015611127, 0.015611127, 0.0074811117, -0.0025379, 0.04200404, 0.018793251, 0.00888499, 0.018418884, -0.0047139116, 0.002516062, 0.014375715, 0.0019342324, 0.0066075874, 0.019417198, 0.014413151, 0.008030185, 0.01187993, -0.028701512, 0.02815244, 0.01280337, -0.024209103, 0.021663403, 0.0042303535, 0.034466773, -0.006198903, -0.02331062, 0.012241819, 0.008048902, 0.010993928, 0.0015185284, -0.02712917, 0.016010452, -0.004327065, -0.016659357, -0.0020465427, -0.040556487, 0.003890303, -0.0015037097, -0.019654296, -0.02493288, 0.01604789, -8.20757e-06, 0.008448228, -0.016322426, -0.008011466, -0.019554466, 0.00028682014, 0.02343541, -0.037985828, 0.0042646704, 0.039433382, -0.033518374, -0.019404719, -0.0059867613, -0.0034597802, 0.0030323772, -0.028726472, -7.550721e-05, -0.026530182, 0.012179425, 0.023335578, 0.006751095, 0.001026391, -0.020702526, 0.024096793, -0.03426711, -0.016609441, -0.008573017, -0.027877904, 0.003587689, -0.042952437, -0.02655514, 0.031347044, -0.001984148, -0.025544347, 0.04627183, -0.0046764747, 0.008947385, 0.0039308593, 0.022898816, -0.0069757155, -0.005300421, -0.0022586843, 0.006563911, 0.0074998303, 0.0045984816, 0.0081487335, -0.024271498, -0.02932546, 0.012716018, 0.026929507, 0.013913995, -0.033892743, 0.0015512856, 0.012085833, -0.005746542, 0.052561205, -0.0024661464, 0.012728497, -0.0019669896, -0.020814836, 0.030298814, 0.0037592743, 0.044724446, -0.025706572, -0.024159187, 0.003843507, -0.005400252, 0.008710286, 0.0057153446, 0.019055309, 0.012765934, 0.02630556, 0.008417031, -0.04614704, 0.010369982, 0.02493288, 0.0136145, 0.031596623, -0.0029809016, -0.035490043, 0.0053472165, -0.00043598222, 0.006788532, 0.0043582623, 0.0456978, -0.0053160195, 0.024845527, -0.02238718, -0.039682962, 0.013489711, 0.012104551, 0.017283302, -0.00519123, 0.002838954, -0.00031080307, 0.0022571245, -0.016422257, 0.021039456, -0.015374028, -0.011199829, -0.042952437, -0.032170653, 0.004741989, -0.031696454, 0.017183471, 0.040531527, -0.009296794, -0.029924447, -0.0161602, 0.009396626, -0.005980522, -0.026280602, 0.02146374, 0.013065428, 0.016784146, -0.024371328, -0.01685902, -0.010132882, 0.010812983, -0.0062581776, -0.0318462, -0.0030604547, 0.003416104, -0.009309273, 0.0006535834, -0.012990555, -0.02757841, -0.010245192, -0.029450249, -0.026729845, 0.010082967, -0.017146034, -0.010164079, 0.011836254, -0.00031177799, -0.008404552, -0.05495716, -0.007836761, -0.006638785, 0.02782799, 0.04420033, -0.006033557, 0.0094652595, 0.014263404, 0.0019279929, -0.009746036, 0.003968296, 0.023273183, -0.029375374, 0.0024957838, 0.037511628, -0.017021244, -0.04524856, -0.008785158, 0.026505223, -0.007006913, 0.0037124783, -0.000380802, 0.012622426, 0.0071254624, -0.0064204037, -0.006401685, -0.012840807, 0.007375041, -0.016010452, -0.003909021, -0.008922427, -0.0019560705, -0.011461887, -0.019816523, -0.004277149, -0.012198143, -0.03601416, 0.03266981, -0.035490043, 0.005677908, -0.023260705, -0.02989949, 0.033992574, -0.004314586, 0.018144349, -0.04442495, 0.0034441815, 0.005053962, 0.025556825, -0.03139696, 0.020789878, -0.017807417, 0.027104212, 0.018531194, 0.013127822, 0.008154973, -0.009795951, 0.0010794265, -0.010644518, 0.03544013, -0.007886677, -0.027753115, -0.011199829, -0.019941311, -0.009664923, 0.02157605, 0.024895443, -0.0066512637, 0.010987688, -0.05850117, 0.015873184, 0.016422257, 0.0061801844, 0.008417031, 0.02538212, 0.00048667783, -0.02410927, -0.033069134, -0.0030495357, 0.002698566, -0.008966103, 0.0005042263, -0.015511296, 0.025107585, 0.059349738, 0.034017533, -0.02259932, -0.012266777, -0.018131869, -0.006070994, 0.026405392, -0.00013297847, -0.020602696, 0.004489291, 0.013102865, 0.008417031, 0.029350417, -0.021713318, -0.017033724, 0.0129531175, 0.030423604, 0.007911635, -0.017745022, -0.015174365, 0.02598111, 0.0135271475, -0.03464148, -0.01085042, 0.035190552, 0.014725124, -0.006969476, 0.044100497, -0.017245866, 0.0256317, 0.026180772, -0.011268464, 0.017932206, -0.025756488, -0.020340638, 0.021950418, -0.0047263904, -0.024508595, -0.008785158, 0.0025924954, -0.040905897, -0.0030292575, -0.010189037, -0.01235413, -0.018294094, 0.014512982, 0.003784232, 0.005169392, -0.004648397, 0.020253286, -0.0041274023, 0.0042428323, 0.043751087, 0.001119203, -0.0228489, 0.012516355, 0.031671494, 0.030448562, 0.033867784, 0.0023070402, -0.01073187, -0.030024279, 0.004779426, 0.024259018, -0.011262224, 0.0057309433, 0.02215008, 0.005122596, 0.019766606, 0.0014826516, 0.05251129, -0.044350076, -0.014113657, -0.018518716, -0.009727317, 0.00090316177, 0.037237093, -0.0071254624, 0.0010591482, -0.015211802, 0.017008767, 0.014238446, 0.008136255, -0.013864079, -0.029924447, -0.0064890375, -0.026180772, -0.005512562, 0.029175712, 0.0055593583, 0.038634732, -0.019779086, -0.052461375, -0.01638482, 0.0047326297, -0.004508009, -0.004096205, 0.019729171, -0.0113059, -0.006211382, 0.02747858, -0.0046702353, -0.009827148, 0.0017548479, -0.008666609, 0.037461713, 0.013639458, -0.0012861085, -0.0039558173, 0.044699486, -0.037811123, -0.039882623, 0.031122424, -0.016834062], "content_hash": "11e5dbff75922f24aa696d75de40c670"}, {"content": "token chips**: Your input tokens are displayed as colored boxes. Darker blue = higher influence. Hover over any chip to see the exact attribution score. - **Horizontal bar chart**: Shows the same information as bars, with attribution scores labeled. Longer bars = more influential tokens. ### What Attribution Scores Mean\n\n- **High score (near 1.0)**: This token was highly influential for the prediction. It strongly pushed the model toward the target token.\n- **Low score (near 0.0)**: This token had little influence on this particular prediction.\n- **Scores are normalized**: The highest-influence token gets a score of 1.0, and others are scaled relative to it.\n\n### Tips\n\n- Compare attribution for different target tokens. You might find that different input tokens drive different predictions.\n- Try Integrated Gradients for the most reliable results, especially when you want to draw conclusions.\n- Short prompts give cleaner attribution results since there are fewer tokens to compare.\n- Function words (like \"the\" or \"is\") often have low attribution; content words (nouns, verbs) tend to have higher attribution.", "source_file": "attribution_panel_guide.md", "chunk_index": 1, "embedding": [0.004626793, -0.0030052688, 0.001941866, 0.002912799, 0.04863912, -0.040158316, 0.0031357172, 0.0076221544, -0.025640557, -0.0037120024, 0.00970933, 0.01939224, -0.046763305, -0.046287745, -0.013751581, -0.00039650558, 0.0019666348, -0.040581036, -0.031413317, 0.034425188, -0.04285315, -0.0072786952, 0.045865025, 0.026908714, 0.020211259, 0.013540222, -0.023883631, -0.026076486, 0.010303779, 0.03949782, 0.0009998298, -0.014213931, -0.008143948, -0.046287745, -0.016301107, -0.021268057, -0.04372501, 0.010601004, 0.029775279, -0.004078579, -0.030937755, 0.028586382, -0.03450445, 0.0042734263, -0.045627248, -0.009597045, 0.024781909, -0.07540253, -0.04282673, 0.09099029, -0.056168806, 0.020369777, -0.021941764, 0.05344755, -0.027450323, 0.047370963, -0.015957648, 0.01882421, -0.027027605, -0.045706507, -0.023157082, -0.026882295, 0.04885048, 0.006350694, 0.027978722, -0.027688103, -0.059127837, 0.01453097, -0.0031621372, -0.016842715, 0.05336829, 0.01877137, 0.050171476, -0.016697405, -0.01955076, -0.014346031, -0.015772708, 0.021122746, -0.0026865783, -0.018309023, 0.008084503, 0.015851967, -0.027899463, -0.024821538, 0.009597045, -0.035746187, -0.03503285, -0.04855986, -0.023910051, -0.052232232, -0.03796546, -0.010528348, -0.027344644, 0.015429248, 0.0026073183, -0.0043493835, -0.035376307, -0.0065323315, -0.010422668, -0.013183553, 0.015217889, -0.020990647, 0.017357904, -0.022245593, -0.0030779236, -0.011664406, 0.001821325, -0.050964076, -0.005888345, -0.014781959, -0.0005094096, 0.051598154, -0.08982781, 0.050224315, -0.002926009, -0.03992054, -0.04282673, 0.0381504, 0.004098394, -0.028850581, -0.017133335, -0.045019586, 0.014121461, 0.03416099, 0.0060336553, 0.0007409969, 0.006112915, -0.0119682355, -0.008368518, -0.06784642, -0.017582474, 0.0019055386, -0.009649885, 0.008553457, -0.014081831, -0.02421388, 0.008850682, 0.04380427, -0.07693488, 0.022866463, -0.011532307, -0.02468944, -0.017450374, -0.0039398745, -0.048216403, 0.012351325, -0.027608844, 0.022787204, -0.037833363, 0.05902216, 0.017146545, -0.0051684016, -0.03759558, -0.005924673, -0.03416099, -0.022351274, 0.012972194, 0.008209998, -0.037278544, -0.051730253, -0.01438566, 0.022787204, -0.00056802883, -0.020832127, -0.037463482, 0.04031684, -0.03291925, -0.017899513, -0.040237576, -0.029326139, -0.036485944, -0.029563919, -0.015825547, -0.057754003, -0.002504941, -0.027582424, -0.050567776, -0.018705321, 0.020990647, -0.011651196, 0.0037780523, 0.02015842, -0.010653843, 0.004986765, -0.01462344, -0.0004916587, -0.030699978, 0.033130612, 0.016419996, -0.0018196738, 0.042958833, 0.013394913, 0.067000985, 0.01994706, -0.016882345, -0.05376459, 0.041135855, 0.032655053, 0.009649885, -0.0007537941, 0.011083167, 0.028242921, 0.02472907, -0.013183553, -0.032628633, -0.018533591, -0.0028203293, -0.04425341, -0.06763506, -0.00045161595, 0.017952353, -0.045415886, 0.028903421, -0.01449134, -0.09796516, 0.0044682734, -0.005627448, 0.04409489, 0.036485944, 0.016763456, -0.0034643153, 0.03479507, 0.04401563, -0.027820203, -0.05328903, 0.032549374, 0.02402894, -0.015970858, 0.050145056, -0.023434492, 0.02892984, 0.031360477, -0.0014861219, -0.057278443, -0.01463665, -0.023328813, 0.022192754, -0.0111360075, 0.04509885, -0.09030338, -0.009352661, -0.008038268, 0.051439635, 0.021571886, 0.00965649, 0.00965649, -0.022892883, -0.027371064, 0.0044319457, 0.008322283, 0.0385467, -0.0239761, -0.022866463, 0.022615474, 0.030515037, -0.017965563, -0.0062119896, 0.04797862, 0.047265284, -0.00982822, 0.0025842008, -0.0064960043, -0.07392301, -0.03323629, -0.01484801, -0.0016826203, -0.02451771, 0.0287449, -0.055006325, -0.028639222, -0.0050660246, -0.003107646, 0.03331555, -0.0054656263, -0.027767362, 0.0027707918, -0.034398768, 0.020211259, -0.040818814, -0.00032508918, -0.01435924, 0.0070343106, 0.059497718, 0.025997227, -0.037120022, -0.013910102, 0.026393525, 0.027767362, 0.005617541, -0.0033322156, 0.018388283, -0.031439736, 0.0394714, -0.035429146, -0.033077773, 0.015587768, 0.017740993, -0.0031984646, -0.022034235, -0.050065797, 0.0129986135, -0.040713135, -0.015165049, -0.023183502, 0.015640607, -0.007873144, 0.040079057, 0.042958833, 0.01957718, -0.04457045, -0.020567928, 0.015138629, 0.005686893, 0.02430635, -0.06589135, 0.040237576, 0.06721234, -0.019616809, 0.03437235, 0.022126704, -0.03756916, -0.055323366, 0.018414702, -0.00062540965, -0.0057067084, -0.011677616, -0.009980135, 0.017384324, 0.004306451, -0.020871757, 0.0019699372, 0.046340585, -0.001429154, 0.047450222, -0.07899564, -0.022853253, -0.027925882, 0.015627397, 0.005650566, 0.060713034, -0.031545416, -0.055164848, -0.0012846699, -0.01223904, 0.03408173, -0.001510065, -0.0031571835, 0.026961554, 0.056010284, -0.007919379, 0.016895555, 0.0071465955, 0.0028368416, -0.029880958, 0.016274687, 0.037833363, -0.011301132, -0.00016006772, -0.017001236, 0.013778002, -0.017106915, 0.005036302, 0.049220357, 0.017357904, 0.0086525325, -0.055112008, -0.025455618, 0.0075494996, 0.03286641, -0.035508405, -0.020700028, 0.009240376, -0.027556004, -0.051413216, -0.031149115, 0.020620767, -0.0006452246, 0.035508405, -0.021624725, -0.0012896236, 0.045653667, -0.056115966, 0.014332821, -0.017542845, 0.051941615, 0.009042227, 0.010898228, -0.004365896, -0.02431956, 0.027503163, -0.0012615524, -0.0078004887, -0.018956311, 0.032760732, -0.004451761, 0.04937888, 0.006254922, 0.043962788, 0.027582424, 0.04319661, -0.053949527, -0.023077823, -0.057331283, -0.017146545, 0.008163763, 0.06789926, -0.005769456, 0.013500593, 0.022734363, 0.062086873, 0.069854334, -0.032179493, 0.015759498, -0.041056596, 0.011268107, 0.021651145, 0.021783246, 0.021479417, 0.013256208, 0.040026218, -0.009313031, -0.056908563, -0.017661734, -0.059074998, -0.007291905, -0.020845337, 0.037120022, -0.011406812, 0.040184736, -0.03817682, 0.017542845, 0.031783193, 0.026168955, -0.036116067, -0.012575895, -0.026076486, 0.060395997, 0.023526961, -0.010171679, -0.0064794915, 0.018811, -0.015019739, -0.02383079, -0.06182267, -0.025032898, -0.02422709, -0.010713288, -0.0676879, 0.005383064, -0.0128004635, -0.028057983, 0.031360477, -0.0047721025, 0.075032644, -0.036512364, -0.04414773, 0.035006426, 0.016472837, 0.026050067, -0.008540248, 0.015997278, -0.012509844, -0.0032430484, 0.017371114, -4.8995582e-05, -0.0035270627, -0.027159704, -0.013593062, 0.03870522, 0.006426652, 0.007952403, -0.022443743, -0.002691532, -0.016023697, 0.025970807, -0.036565203, 0.032153074, -0.046684045, 0.031386897, -0.03471581, 0.0056901956, -0.0091677215, -0.04317019, -0.014015781, 0.0062780394, 0.05939204, 0.026921924, -0.027186124, -0.013348677, -0.012311694, -0.010581188, -0.04708034, 0.017265435, -0.01475554, 0.021413365, -0.028665641, 0.02409499, -0.014121461, 0.010039579, -0.03949782, 0.057701163, -0.02924688, -0.06647258, 0.038229663, -0.03519137, 0.0025016386, 0.06219255, 0.004441853, -0.04327587, 0.03928646, -0.03915436, -0.032153074, -0.02924688, -0.0067602037, -0.041928455, 0.003857312, 0.009431921, 0.037172865, -0.00180151, 0.006783321, 0.018454332, -0.0009213956, -0.012100335, -0.008639323, 0.009194141, -0.02414783, -0.011717246, -0.044940326, -0.046763305, -0.025587717, 0.014081831, -0.031836033, 0.021637935, 0.018665692, -0.06282663, 0.022390904, 0.0048249424, -0.010105629, -0.036987923, 0.011182242, 0.036882244, -0.014028992, 0.016169006, 0.0030349912, 0.00046152342, 0.010970882, -0.01976212, -0.0381504, -0.0020029622, 0.026855875, -0.016102957, -0.028586382, 0.03513853, -0.017344695, -0.003335518, -0.024121411, 0.030911336, -0.017120125, -0.016221847, -0.005264174, 0.035376307, -0.012575895, 0.016525676, 0.002389354, -0.035640508, 0.028057983, 0.049009, 0.03392321, 0.032707892, 0.0031109487, -0.011413417, 0.002222578, 0.020937808, -0.009636675, 0.016023697, -0.0024289838, -0.005845413, 0.013309048, 0.027767362, -0.013091083, -0.0076089446, 0.0108321775, -0.0141478805, 0.0016182216, -0.0013465916, 0.0040653693, 0.009352661, 0.021558676, 0.015244309, 0.03949782, 0.016710615, 0.0032579096, -0.0015108907, 0.016631356, -0.03466297, 0.045521565, 0.014055411, -0.0099999495, -0.011690826, 0.006509214, 0.01495369, -0.031281214, -0.01968286, -0.012628734, 0.008857287, 0.046155646, -0.022892883, 0.019867798, -0.0060567725, 0.015112209, -0.031492576, -0.044781808, -0.01996027, 0.0047754054, -0.0080316635, -0.0048909923, -0.013038243, 0.020739658, 0.021122746, -0.018573223, -0.026697354, -0.031281214, -0.006469584, -0.008725187, -0.012655154, 0.039233617, -0.0022044142, -0.01475554, 0.0056142383, 0.0031720446, -0.028295761, 0.0071730153, 0.026499206, 0.017410744, 0.020449039, 0.011888975, -0.00084254856, 0.01463665, -0.00055069075, -0.017727783, 0.030013058, 0.0054458114, -0.019828169, -0.01974891, 0.015165049, -0.00243724, -0.00054037047, 0.002765838, -0.026882295, -0.00997353, -0.022641893, -0.031783193, 0.010356619, -0.011162427, -0.018692112, -0.03949782, 0.021347316, 0.010508534, -0.0494053, -0.026089696, -0.01210694, 0.010594398, -0.036987923, -0.036248166, -0.01209373, -0.0038176822, -0.007661784, 0.032153074, 0.002452101, -0.011373786, -0.057754003, 0.017159754, 0.0078004887, 0.032734312, -0.0015835455, 0.0036525575, -0.016208638, -0.04858628, -0.031360477, 0.036644466, 0.0054953485, 0.015495298, -0.026684145, -0.050831977, 0.007866539, -0.017120125, 0.027503163, 0.008632718, 0.021492627, -0.029828118, -0.015112209, 0.023011772, 0.0069880756, -0.0015629049, -0.03450445, 0.00019907842, 0.013949731, 0.04951098, -0.011129403, 0.0071069654, 0.004600373, -0.0025891548, -0.021386946, -0.03402889, -0.020435829, 0.019828169, 0.011202057, 0.023038192, 0.008051478, -0.021479417, -0.01992064, 0.026763406, 0.012047495, 0.0037252123, -0.01471591, 0.030911336, 0.004567348, -0.0023051403, -0.014319611, -0.06684246, -0.016367156, 0.001345766, -0.041030176, 0.0025462222, 0.031281214, -0.016301107, -0.013262813, -0.023183502, -0.020435829, 0.0134147275, -0.010171679, 0.0018890261, -0.008890312, -0.02898268, 0.0014885988, 0.01475554, -0.03471581, 0.015759498, -0.011360576, -0.010224519, 0.000545737, -0.008513828, 0.013276023, -0.024980059, 0.023262762, 0.016380366, -0.030752817, 0.027265385, -0.012113545, -0.063724905, -0.018163713, 0.011162427, 0.032020975, -0.00044831345, 0.009114881, 0.0075759194, -0.008586482, -0.047793683, 0.002623831, 0.023130663, -0.016063327, 0.03754274, -0.016921975, -0.0015150188, -0.017912723, 0.005409484, -0.0030036175, 0.017252225, -0.025944388, 0.059550557, -0.0126947835, -0.015653817, 0.038335342, -0.016486047, 0.030964177, -0.01461023, 0.016221847, 0.021400155, -0.03342123, -0.0036129276, 0.00981501, -0.022219174, -0.023579802, 0.010997303, 0.010838782, -0.0045277183, 0.014161091, -0.030303678, 0.008434568, 0.008322283, -0.020079158, 0.02383079, 0.017846674, -0.037357803, -0.01998669, 0.008315678, -0.014900849, -0.10235087, -0.023091033, 0.057225604, -0.008566667, 0.008896917, -0.0028071192, 0.03455729, 0.006367207, -0.040263996, 0.03283999, -0.038731642, -0.008295863, -0.028586382, -0.0026964857, -0.0075494996, -0.01877137, 0.0015059368, -0.0028649129, -0.010085815, 0.011756876, -0.027397484, 0.002351375, 0.0055283736, 0.006895606, -0.036380265, 0.008005244, -0.03416099, 0.0064960043, -0.0025363148, 0.0044682734, -0.0107264975, 0.0014481433, 0.039682757, 0.01945829, 0.008731792, 0.010112234, -0.0035270627, 0.015085789, 0.0009469899, -0.023064613, 0.027265385, -0.024834748, 0.023091033, -0.015231099, 0.077886, -0.07693488, -0.025733028, -0.022747573, 0.00038350202, 0.012503239, -0.0038143797, 0.017252225, 0.014438501, 0.010779338, 0.0072192503, 0.01900915, 0.029114781, -0.035666928, -0.010541558, 0.009597045, -0.018295811, 0.013659112, 0.0015851967, 0.0020178233, 0.032655053, 0.00977538, -0.0026799731, 0.00600063, -0.026736986, -0.021651145, -0.006664431, 0.01896952, 0.008989386, -0.0021945068, -0.015600977, 0.021215217, 0.010350014, -0.0010402853, -0.054319408, 0.01939224, 0.056908563, -0.009293216, 0.016380366, 0.031994555, 0.014227141, 0.046129227, -0.0035336677, 0.0064596767, 0.0047588926, 0.017278645, 0.056221645, -0.005865228, 0.0018939798, 0.013870471, -0.021756826, -0.015548138, 0.035904706, 0.03756916, 0.022285225, -0.008467592, 0.013025033, -0.0009866199, -0.029537499, -0.01446492, -0.0095376, -0.017225804, -0.01997348, -0.04813714, -0.0192073, 0.0070475205, 0.003219931, 0.030145157, -0.0034676178, 0.013804422, -0.017833464, -0.002750977, -0.0022159729, 0.022377694, 0.027291805, -0.006611591, 0.0022721153, 0.011479466, -0.018916681, 0.0023943076, -0.009029017, 0.04858628, 0.0023216528, -0.008157158, 0.008527038, -0.014095041, 0.014887639, 0.005108957, -0.016353946, -0.020884966, -0.03331555, 0.005029697, 0.043090932, -0.029933799, -0.054794967, 0.020911386, -0.004425341, 0.05239075, 0.025759447, -0.009128091, -0.061135754, -0.016499257, -0.04036968, -0.027344644, 0.021003857, 0.013976151, 0.018480752, 0.014808379, -0.009200746, 0.014068621, -0.01438566, 0.0132165775, 0.020079158, -0.024359189, 0.0070805456, 0.063619226, 0.018256182, 0.030594297, 0.030092318, 0.008157158, 0.043090932, -0.014028992, -0.011294527, 0.020739658, 0.0067403885, 0.0021862504, -0.013011823, -0.016486047, 0.010871808, 0.050620615, 0.004293241, 0.020700028, 0.005941185, 0.011928606, -0.011287922, 0.023857212, -0.01177669, 0.018190132, -0.027556004, 0.0042470065, 0.030832076, -0.0042668213, 0.025482038, 0.0050726295, -0.01908841, -0.0046367003, 0.026750196, 0.027793784, 0.021386946, 0.046419844, -0.03289283, -0.024108201, 0.028903421, 0.0014299796, -0.0062053846, 0.025402777, -0.036036804, -0.011169032, 0.039656337, 0.020211259, 0.024966849, 0.02948466, -0.02853354, -0.012826883, 0.035799026, -0.0022737666, -0.023593012, 0.016697405, -0.0041809566, 0.022800414, -0.017582474, -0.00304655, -0.03405531, -0.0011492677, -0.0024933824, -0.004019134, -0.030937755, -0.017331485, 0.011545517, -0.0024141225, 0.009240376, -0.003910152, 0.046023544, -0.01435924, 0.018044824, 0.017239014, 0.0014052108, -0.005604331, 0.014068621, -0.003929967, -0.009412105, 0.009240376, 0.013672322, -0.004613583, 0.040924497, 0.0074174, 0.027740942, 0.037806943, 0.009715935, -0.013910102, -0.037622, -0.018295811, -0.004488088, 0.026340686, -0.011096377, -0.002389354, 0.0041215117, 0.055957444, 0.041030176, -0.046895403, -0.0066512213, -0.017674943, 0.0492732, 0.023659062, 0.026974764, 0.019828169, 0.016036907, 0.030515037, 0.011961631, -0.027503163, 0.007734439, -0.02453092, -0.01453097, 0.0039828066, -0.006955051, 0.022655103, -0.021915345, 0.031968135, 0.009180931, 0.026142536, 0.0026155745, -0.0034775254, -0.0012970542, -0.041188695, 0.009504575, 0.0028550054, -0.036644466, 0.0052212416, -0.032100234, 0.006948446, 0.023249552, 0.012569289, 0.0075230794, 0.0048447577, -0.013883681, 0.01472912, 0.0021565282, 0.030594297, 0.025349937, 0.0067569013, -0.0036063227, 0.017265435, 0.021043487, -0.0064497693, 0.007899564, -0.014808379, 0.036591623, 0.0071069654, -0.016129376, -0.018599642, 0.008216603, 0.028903421, 0.017582474, -0.020766078, 0.00712678, -0.01198805, -0.027133284, 0.013645902, -0.020673607, 0.032338012, 0.015297148, -0.020131998, -0.015231099, -0.052020874, -0.0064662816, -0.0038441021, 0.0031836035, 0.0015695099, 0.00961686, 0.027106864, 0.018850632, -0.0090686465, 0.00610631, 0.00010537017, -0.011618171, 0.010964277, -0.02441203, 0.0006307762, -0.0027410693, -0.0005222067, -0.038308922, 0.020937808, -0.0033454257, -0.007846724, -0.015310358, -0.054425087, 0.024847958, 0.024781909, 0.0048975972, -0.023024982, 0.036776565, 0.008692162, -0.0057001035, -0.0019121435, 0.0010865203, 0.0039530843, 0.07994676, -0.011202057, -0.0029573827, 0.00972254, -0.009544205, 0.033526912, -0.020198049, 0.0037483298, -0.03236443, -0.005264174, 0.013844051, 0.026287846, 0.03865238, -0.003850707, -0.023645852, 0.0010650541, 0.0074438197, -0.0018923286, 0.026697354, 0.0072126454, 0.026512416, 0.021651145, -0.005739733, 0.0074570295, -0.0054986514, -0.008824262, -0.0037516323, -0.0029904076, 0.020554718, 0.0071730153, 0.00019237022, -0.029828118, 0.00068774424, 0.0120673105, -0.021056697, 0.018335443, -0.032126654, 0.035508405, -0.035323467, -0.016737036, 0.017754203, 0.008157158, 0.00961686, 0.021743616, -0.018335443, -0.017569264, 0.01475554, -0.015019739, -0.0014431896, -0.011162427, -0.0064299544, -0.0017371115, -0.024610179, -0.025376357, 0.015534928, -0.007787279, 0.0053335265, 9.773316e-05, 0.0016784922, -0.02464981, -0.0049306224, 0.0062516197, -0.02903552, -0.008731792, 0.03899584, -0.027714523, 0.010944462, -0.015402828, 0.0012070612, 0.029880958, -0.006677641, -0.00480843, -0.020739658, 0.019590389, 0.021651145, -0.013857261, 0.005538281, 0.012945773, 0.0061756624, -0.028375022, -0.017674943, -0.025415989, -0.03503285, 0.0004041426, -0.035376307, -0.027186124, 0.039603498, 0.009768775, -0.015085789, 0.013738371, -0.015521718, -0.011314342, 0.035957545, 0.016314317, -0.001915446, -0.021241637, -0.0022754178, 0.010165074, 0.008474197, -0.00713999, 0.005571306, -0.017516425, -0.028375022, -0.0075561046, 0.016380366, 0.0107264975, -0.015984068, 0.012879724, 0.020779287, -0.010607609, 0.033976052, 0.00982822, 0.027820203, 0.019365821, 0.0017239015, 0.02404215, 0.012021075, 0.031783193, -0.046023544, -0.02882416, 0.0024917312, 0.002009567, 0.004425341, -0.00986785, 0.012648549, 0.018916681, 0.030884916, 0.0025759446, -0.03479507, -0.017476793, 0.026961554, 0.007813699, 0.04229833, 0.013711952, -0.003287632, 0.005115562, 0.0022671616, -0.0029161016, -0.002840144, 0.02379116, 0.0053203166, 0.016974814, -0.010178284, -0.044781808, 0.02916762, 0.0049768575, 0.018375073, 0.016155796, 0.0066908514, -0.002899589, -0.00081736705, -0.02488759, 0.015640607, 0.0069682607, -0.016301107, -0.026538836, -0.010951067, -0.0056340536, -0.015627397, 0.0145706, 0.01234472, -0.020277308, -0.009451736, -0.011849346, 0.026287846, -0.010072604, -0.050382838, 0.008018454, 0.011829531, 0.0029805002, -0.0021912043, -0.011056747, 0.004002622, 0.042034134, -0.0020723145, -0.01966965, 0.010250939, 0.017635314, -0.021624725, 0.011803111, -0.011710641, -0.026631305, 0.014425291, -0.007886354, -0.0046928427, 0.0100990245, -0.0032314896, -0.012470215, 0.028163662, 0.0018626062, -0.017886303, -0.06763506, -0.0036789775, 0.00044212127, 0.029114781, 0.04908826, -0.02972244, 0.010158469, 0.021439785, 0.01438566, -0.0009461643, 0.0054160887, 0.02464981, -0.020686818, -0.008196788, 0.017437164, -0.020581137, -0.03424025, 0.011010513, 0.007648574, 0.016684195, 0.00028545925, 0.0058289007, 0.0190752, 0.018692112, -0.016486047, -0.006889001, -0.014068621, 0.014319611, -0.014914059, 0.006046865, -0.010640633, 0.016578516, -0.017542845, -0.02850712, -0.017859884, 0.00059444876, -0.020198049, 0.040950917, -0.0011327552, 0.027291805, -0.026367106, -0.027014395, 0.01482159, 0.01246361, 0.016525676, -0.047370963, -0.008764817, 0.000552342, 0.031545416, -0.041717093, -0.0004293241, -0.014887639, 0.014980109, -0.012866514, 0.036485944, 0.012985404, -0.0053467364, -0.00048381527, -0.011743666, 0.02964318, 0.0023794463, -0.021532256, -0.021624725, -0.022100285, -0.03849386, 0.007252275, 0.014028992, 0.0045211134, 0.005693498, -0.07719908, 0.00019763358, 0.02354017, -0.010264149, 0.010118839, 0.013844051, -0.012324905, 0.012853304, -0.03381753, 0.0034841304, -0.016182216, -0.0020871756, -0.006231805, -0.021294476, 0.007846724, 0.03519137, 0.03326271, 0.0018824211, -0.0066512213, -0.03426667, 0.0013309048, 0.007787279, 0.0005973385, -0.013857261, 0.0095838355, 0.00022477594, 0.0046367003, 0.04229833, 0.0047688, -0.013091083, 0.0016495953, 0.015957648, 0.0015868479, -0.005970908, -0.02453092, 0.012562684, 0.01896952, -0.021083117, -0.0013276023, 0.018850632, 0.041267954, -0.0064134416, 0.023157082, 0.002691532, 0.02426672, 0.028639222, 0.017727783, 0.020581137, -0.022007816, 0.0043229638, 0.009729145, 0.00024748058, -0.016433205, -0.016129376, 0.00053995766, -0.026168955, 0.001553823, -0.008606297, -0.011789901, -0.0050495123, -0.00952439, -0.026776616, 0.009464946, 0.009577231, 0.022166334, -0.0019501222, 0.0013350329, 0.055534724, -0.008427963, 0.0036723725, 0.012833488, 0.0035666926, 0.026023647, -0.0009337799, 0.021439785, -0.022681523, -0.01204089, -0.0013886984, 0.008176973, -0.024557339, -0.014914059, 0.017080495, 0.009299821, -0.0015596024, 0.0021202005, 0.030594297, -0.017225804, -0.013203368, -0.037859783, -0.0144517105, 0.03426667, 0.03402889, 0.009088461, 0.0027377668, -0.0039167567, 0.027476743, 0.03373827, -0.016406786, -0.0082364185, -0.041875616, 0.010924648, -0.026459575, -0.01956397, 0.029775279, 0.016182216, 0.030699978, -0.015970858, -0.048242822, 0.004019134, 0.0033751482, -0.006822951, 0.008639323, 0.010904833, 0.0034511054, 0.0008359436, 0.036538783, -0.016697405, -0.0043196613, 0.00984143, -0.021360526, 0.03997338, 0.016419996, -0.011472861, -0.003454408, 0.022998562, -0.015627397, -0.04832208, 0.015125419, 0.013394913], "content_hash": "903501e06932bc125329a18df29e142e"}, {"content": "# Beam Search and Generation\n\n## What Is Text Generation?\n\nWhen you click \"Analyze\" in the dashboard, the model generates a continuation of your prompt. The simplest approach is **greedy decoding**: at each step, pick the single most likely next token. This is fast but can lead to repetitive or suboptimal text.\n\n## What Is Beam Search?\n\nBeam search is a smarter generation strategy that explores **multiple possible paths simultaneously** instead of committing to the single best token at each step.\n\n### How It Works\n\nImagine you're at a fork in the road. Greedy decoding always takes the path that looks best right now. Beam search keeps multiple paths open and evaluates them over several steps before deciding which one is actually best overall.\n\nMore specifically:\n1. At each step, the model considers the top candidates for the next token\n2. It keeps the **N best partial sequences** (where N is the number of beams)\n3. It extends each of those sequences by one more token\n4. It ranks all the possibilities and keeps the top N again\n5. This continues until the desired number of tokens have been generated\n\nThe result is that beam search can find better overall sequences, even if the individual tokens at each step aren't always the highest-probability choice.\n\n### Example\n\nWith the prompt \"The cat sat on the\" and 3 beams:\n- Beam 1 might generate: \"The cat sat on the mat and purred\"\n- Beam 2 might generate: \"The cat sat on the floor and looked\"\n- Beam 3 might generate: \"The cat sat on the couch and slept\"\n\nEach beam represents a different possible continuation.\n\n## Generation Controls in the Dashboard\n\nIn the generator section, you have two settings:\n\n- **Number of Generation Choices (Beams)**: How many parallel paths to explore (1-5). With 1 beam, you get greedy decoding. More beams = more diverse results but slower.\n- **Number of New Tokens**: How many tokens to generate beyond your prompt (1-20). Longer generations take more time.\n\n## How Generated Sequences Are Used", "source_file": "beam_search_and_generation.md", "chunk_index": 0, "embedding": [0.021287892, 0.009334653, 0.033566758, -0.0059562684, -0.0076997317, -0.030554706, -0.007604757, 0.030364757, 0.024164269, 0.036497407, 0.0012177109, -0.021233622, -0.031260233, -0.046103414, 0.009009025, -0.023811504, -0.035737608, -0.017597448, 0.02470698, 0.068598844, 0.009192191, -0.021884875, -0.007028125, 0.038234085, -0.01381203, -0.006332775, -0.0300934, 0.044800907, 0.015290921, -0.020229604, -0.013628865, -0.041327547, -0.022074826, -0.046781804, -0.05877575, 0.0027457853, -0.038125545, 0.009409276, 0.016498456, -0.020962264, -0.026172305, -0.010033395, -0.004365443, -0.0050675767, -0.011600477, 0.012570575, -0.020270307, -0.053185813, 0.014870319, 0.0383969, 0.031314503, 0.055899374, -0.023513013, -0.031612996, -0.0021589776, -0.015779361, -0.06637372, -0.02359442, -0.009368572, -0.02006679, 0.032291386, -0.0013347332, 0.01839795, 0.07777068, 0.025602456, -0.006482021, -0.025507482, 0.019049205, -0.015223082, 0.02508688, 0.0144768525, 0.019605484, -0.018899959, 0.0024031983, -0.045289345, 0.0061055142, 0.030554706, 0.010786409, -0.027895415, -0.018140161, 0.038505442, 0.0025812758, 0.0006639747, -0.048518486, -0.0120007275, -0.00894797, 0.02470698, -0.0016866482, -0.034462236, -0.045533568, -0.05796168, 0.020636637, -0.047731552, 0.05845012, 0.014001979, 0.009280382, -0.0060037556, -0.046998892, 0.0070688287, 0.008473096, 0.03622605, -0.022427589, -0.040622022, -0.01290977, 0.05299586, -0.010440429, -0.021369299, 0.0056984797, -0.024204971, -0.045804925, -0.05085215, -0.02229191, 0.005773103, 0.016010014, 0.022508996, 0.010467565, 0.0043790103, 0.0221698, 0.022956733, 0.008975105, -0.04227729, 0.006763553, -0.023404472, -0.016023582, -0.012536655, -0.0021216662, 0.024530599, -0.048409943, -0.013337157, 0.023689395, 0.033458218, -0.0036056454, 0.019117042, -0.04043207, -0.05920992, 0.02873662, -0.022251207, 0.00993842, -0.03760997, 0.037365746, -0.0013135335, -0.0081271175, -0.0048437077, -0.010956006, -0.051557675, -0.013907004, -0.0149517255, 0.024747685, -0.030934604, 0.015060268, 0.029333603, -0.032617014, -0.05877575, -0.040730562, 0.046103414, -0.016457753, -0.011159523, 0.05573656, -0.06262901, 0.004073735, -0.008194957, 0.033593897, -0.014409013, -0.009850229, -0.056062188, 0.026606474, -0.019551214, 0.032101437, -0.022061257, -0.022156233, 0.0158472, -0.021138646, 0.009219326, 0.0038159464, 0.04477377, -0.016498456, -0.084228955, -0.060729515, -0.024679845, -0.050580792, -0.014327606, 0.013432131, -0.030201944, 0.0010328494, 0.0053660683, -0.0048742355, 0.005606897, -0.0005906237, -0.005094712, 0.0024371177, 0.0021827214, 0.014069818, 0.074243054, -0.007815058, -0.040106446, -0.03541198, 0.06328026, 0.022658242, 0.01796378, -0.021789901, 0.05573656, -0.042114478, 0.023607988, 0.005064185, -0.024489895, 0.0336753, -0.031857215, -0.0012397586, -0.04187026, -0.02222407, 0.028763756, -0.02272608, 0.049332555, -0.0022980478, -0.043362718, -0.01659343, 0.026280846, 0.03467932, 0.002208161, 0.041408952, -0.0018401342, -0.0128962025, -0.004894587, 0.011037413, 0.018669305, 0.058178764, 0.0074758627, -0.035140626, 0.020704476, 0.014096954, 0.060512427, -0.010528619, 0.011722587, -0.021884875, 0.001067617, -0.028221043, -0.032345656, 0.009999475, 0.017244685, -0.02965923, -0.011478366, -0.008398474, 0.035819016, -0.00483014, -0.0112884175, 0.030744655, -0.07066115, -0.02941501, 0.012468817, 0.064257145, 0.015874337, 0.009965556, -0.008975105, 0.030066265, 0.018791415, -0.010379374, -0.035574794, 0.029007975, -0.026023058, -0.00668893, 0.008038927, 0.025290396, -0.03101601, -0.0038736095, -0.049305417, 0.009965556, -0.028193908, -0.077173695, -0.044692364, -0.035656203, -0.02440849, -0.033349674, -0.010223344, -0.017516041, -0.033973794, -0.051503405, 0.0032325306, -0.012034646, 0.0005274486, 0.0069331503, -0.034896404, -0.009606008, 0.049006928, 0.008751237, -0.010847463, 0.0058104144, -0.0018706616, 0.018479357, 0.03874966, -0.017814534, 0.0006974702, 0.00050158496, 0.0015882817, -0.006454885, -0.007082396, 0.040214986, -0.0005070969, -0.007048477, -0.035764746, 0.021206485, 0.029089382, -0.04515367, 0.02477482, -0.04024212, -0.007116316, 0.0028814634, -0.04045921, 0.021640655, -0.004484161, -0.014436149, -0.004619839, 0.034842134, -0.03771851, -0.07326617, 0.018954229, 0.0019537646, 0.051259182, -0.028899433, 0.034923542, -0.038478307, -0.0013771326, -0.015928607, 0.020541662, -0.02154568, -0.026307981, -0.0040058955, 0.008934402, -0.0092328945, -0.0035852937, -0.026321549, 0.022346182, 0.019239154, 0.025466777, 0.05573656, -0.036985848, -0.074351594, 0.011566557, 0.027108483, 0.020365281, 0.023445174, -0.024571301, -0.003768459, -0.016010014, -0.04347126, -0.013642432, -0.0020656988, -0.012292435, 0.02694567, 0.022373317, 0.006397222, -0.02712205, -0.011912537, -0.0032613622, -0.029767774, 0.039618004, 0.013683136, -0.0004189061, -0.023336632, -0.022346182, -0.009673848, -0.017516041, 0.040025037, 0.013201478, 0.023458742, 0.00017267157, -0.037501425, -0.009897716, -0.008011791, 0.020609502, 0.015453734, -0.02248186, 0.04298282, 0.0039550164, 0.018818552, -0.04409538, 0.013628865, -0.04173458, 0.0253311, -0.0011210402, 0.08878774, -0.00866983, -0.027366271, -0.055628017, -0.043525532, 0.020650206, 0.05568229, -0.0050981045, 0.0050200894, -0.036198914, -0.000110768444, -0.036497407, -0.0387768, 0.021749198, -0.011152739, 0.0019673323, 0.049658183, 0.0030239255, 0.029116519, 0.037935596, 0.030419027, 0.042195886, -0.025263261, -0.049685318, -0.011254498, -0.018099457, 0.0038532577, 0.018845687, 0.03820695, 0.04762301, 0.0021521938, 0.042820007, 0.032779828, 0.039156698, -0.04005217, 0.029170789, 0.036253184, 0.040214986, 0.04162604, -0.00047402532, 0.053728525, -0.021681359, -0.077065155, -0.03598183, -0.021884875, -0.038369764, 0.023892911, 0.0033749924, 0.0070688287, 0.019727595, -0.024137132, 0.0058782534, 0.044068243, 0.07896465, -0.009124352, -0.012713037, -0.0024422056, 0.058558665, 0.029116519, -0.004769085, -0.04290141, 0.016688405, -0.0063565187, -0.038613986, 0.0011507198, -0.032671284, 0.010738921, -0.03183008, -0.023974318, -0.004124614, -0.032128572, -0.021287892, 0.02321452, -0.021776333, 0.026267279, -0.04366121, 0.020175332, 0.064311415, -0.016484888, 0.04941396, 0.0024710372, 0.02898084, -0.013350724, 0.00039346647, -0.0072384262, -0.006875487, 0.02199342, 0.002839064, 0.019944679, 0.021206485, -0.007821842, 0.004752125, 0.016064286, -0.00032499142, -0.047161706, -0.020134628, -0.00040003838, 0.016579863, 0.005905389, 0.0021572816, -0.026986372, 0.019184882, 0.05790741, 0.03318686, 0.07608827, -0.017475339, 0.072560646, 0.01907634, -0.015928607, 0.018994933, -0.024042157, -0.012285651, -0.01883212, 0.023173818, -0.015942175, 0.010108017, -0.021681359, -0.032101437, 0.0241507, 0.01381203, 0.012875851, -0.0034665752, -0.001353389, -0.017841669, 0.03448937, 0.028845163, -0.02303814, 0.04284714, 0.039102428, 0.0030612368, 0.01468037, -0.019673323, -0.02583311, 0.011783642, 0.019483374, -0.008350986, 0.03394666, -0.01765172, 0.04501799, 0.007095964, 0.014205496, 0.017841669, 0.021572817, 0.014409013, 0.0052643097, 0.00038816655, -0.060512427, -0.053240083, -0.059861172, -0.034977812, 0.0002633003, 0.036334593, -0.043742616, 0.0054440834, 0.04105619, -0.06545111, -0.05269737, -0.011261282, 0.030364757, -0.012428113, -0.024883362, -0.027312, 0.021586385, -0.02291603, 0.02402859, -0.005094712, -0.0026966021, 0.013018313, 0.006129258, 0.0059087807, 0.007502998, 0.02291603, -0.037012983, 0.041571766, -0.03815268, -0.018520059, 0.020216035, -0.02006679, -0.022997437, 0.02463914, -0.0253311, -0.010304751, 0.0150467, 0.020595934, -0.003917705, -0.048762705, 0.016946193, -0.0029001192, 0.023743667, -0.009178623, 0.0097620385, -0.0042772517, -0.002067395, -0.0052337823, 0.042440105, 0.014341175, -0.013330372, 0.0057968465, 0.008907267, -0.024734115, 0.029550688, -0.016824083, -0.01683765, 0.0062344084, 0.012278867, -0.04284714, -0.0005261766, -0.0154265985, 0.035004947, 0.013601729, 0.008493449, 0.013113287, 0.004484161, 0.0038091624, 0.030066265, -0.03419088, -0.01021656, -0.02161352, -0.033349674, -0.025249694, -0.0079236, 0.024299946, -0.041517496, -0.047460195, 0.01591504, -0.02446276, 0.0044264975, -0.017366795, 0.022685377, 0.025670296, -0.021477843, -0.03372957, -0.0030137496, -0.030066265, -0.032752693, 0.0075640534, 0.011532637, -0.007855762, 0.0015407943, 0.027610492, -0.02538537, 0.025724566, 0.023580853, 0.023987886, -0.008914051, -2.5095147e-05, -0.012753741, 0.0008759717, -0.03128737, 0.011742939, 0.033241134, 0.0071705873, -0.011132387, 0.0099452045, -0.008262795, -0.0069467183, 0.0443396, 0.027746169, 0.02253613, -0.010426861, -0.0013703487, 0.012651982, 0.0062988554, -2.1848942e-05, -0.030826062, -0.020446688, 0.024218539, -0.0040296395, 0.017203981, -0.010942438, -0.034353692, -0.037447155, -0.028790891, 0.051177777, -0.022373317, 0.04732452, -0.01598288, -0.004026247, -0.009341436, 0.004684286, -0.017733127, -0.007502998, 0.011960024, -0.028465264, -0.051883303, -0.0075640534, 0.05839585, 0.009823094, 0.024544166, -0.0070145573, 0.0027271297, -0.040757697, 0.039726544, 0.0019741163, -0.004765693, -0.035683338, 0.0056645605, 0.010827112, -0.03541198, -0.0060139312, 0.009076864, -0.004192453, 0.00439597, -0.039618004, -0.02724416, 0.04873557, -0.009585657, 0.02606376, -0.004446849, 0.037040118, 0.018316543, 0.009253246, 0.0360361, -0.016322074, 0.01683765, 0.00033219933, -0.03864112, 0.0053423247, 0.03264415, 0.006892447, -0.077010885, -0.0423587, 0.009619577, 0.0069467183, -0.013194694, 0.0033834723, -0.021315029, -0.039835088, 0.03760997, -0.010915303, -0.0020996185, -0.010453997, -0.03535771, -0.0069161905, 0.018804984, 0.0190085, -0.007604757, -0.024720548, -0.017258253, 0.017176846, -0.0079168165, -0.036008965, 0.021057239, -0.036823034, -0.02965923, 0.0061224736, -0.025928084, 0.008568072, -0.0061496096, -0.039590865, 0.04892552, 0.030853197, -0.0037718508, 0.00035763896, 9.2801696e-05, -0.012523088, 0.010270831, -0.031043148, 0.00014786792, -0.010108017, -0.01579293, 0.014734641, 0.0064989803, 0.025195422, 0.0028271922, 0.0009395708, 0.001846918, 0.0035072786, 0.02341804, 0.012570575, -0.01164118, -0.0069467183, 0.011437663, -0.005288054, -0.04596774, 0.06463704, 0.018316543, 0.04732452, -0.0021928973, 0.0027440893, 0.03728434, 0.013506754, 0.0018248704, -0.03809841, 0.030229079, -0.0054440834, -0.023702962, 0.0025999313, 0.003122292, -0.007408024, -0.0004121222, -0.007631893, 0.0072859135, -5.596721e-05, -0.0006080074, 0.047351655, -0.028926568, 0.060349613, 0.0069467183, -0.050825015, 0.0016925841, 0.019971816, 0.020880857, -0.036958713, 0.008317067, -0.010426861, -0.0031748672, -0.0018113025, -0.028600942, -0.031368773, -0.026090898, -0.0051693353, -0.020948697, 0.049820997, -0.004819964, -0.021206485, 0.0049522505, 0.012312787, -0.112232916, -0.023105979, 0.009171839, 0.014042682, 0.0028967273, -0.028193908, 0.038668256, 0.015969312, -0.006159785, 0.0013389732, -0.0012024471, -0.016729109, 0.043335583, 0.0064752367, 0.0047182054, -0.039970767, -0.008622343, -0.007930384, 0.019103475, -0.007869329, -0.022142664, 0.01999895, -0.013303237, -0.009795958, -0.016145693, -0.020840155, -0.011159523, 0.004979386, 0.023987886, -0.031260233, -0.010677866, 0.07473149, -0.013567809, 0.018994933, 0.0069229747, 0.0014492116, -0.007469079, -0.0063768704, 0.00938214, -0.005684912, -0.020528095, -0.020962264, 0.021627087, -0.009836662, 0.0016434009, 0.0063158153, -0.004399362, 0.0055831536, 0.0027457853, 0.012808012, -0.00934822, 0.00028916393, -0.0026423307, 0.00084629207, 0.018180864, -0.0037888107, 0.01080676, 0.01405625, 0.035086356, 0.045614973, 0.010738921, 0.02192558, 0.031314503, -0.024734115, 0.011023845, -0.02718989, 0.010074099, 0.0018129984, -0.026077328, 0.009056512, -0.013201478, 0.010983141, -0.004046599, -0.0008183085, -0.00096840237, -0.009009025, 0.028600942, 0.0071366676, -0.0071705873, -0.008812292, 0.028628077, -0.010847463, -0.01653916, 0.031450182, -0.017488906, 0.0304733, -0.009857013, -0.008710533, 0.011980375, 0.019212017, 0.022644673, -0.05245315, -0.007184155, 0.0035988614, -0.049658183, -0.014856751, 0.0024031983, 0.020690909, 0.030256214, -0.0012270387, 0.026986372, 0.035004947, -0.031992894, -0.027393406, -0.0038736095, 0.016756244, -0.011220578, -0.003208787, -0.033539623, -0.011837914, 0.009477114, -0.0007890529, -0.0043111714, 0.013947708, -0.02649793, 0.033648167, 0.0411376, -0.016267803, 0.004911547, -0.010684649, 0.0241507, -0.017081872, -0.021749198, 0.006461669, 0.017380364, 0.01795021, -0.0006487109, -0.012828363, 0.0053558927, 0.007340185, -0.009158271, 0.007292697, 0.031612996, -0.0026609865, -0.020853722, -0.022115529, -0.0005244806, 0.007197723, -0.0443396, -0.010847463, -0.024951201, 0.003476751, 0.061055142, -0.0018910134, -0.00938214, -0.037257202, -0.023892911, -0.013757759, -0.027976822, 0.013642432, 0.022807486, -0.0020945305, -0.025114015, 0.027651194, -0.037772782, 0.02470698, -0.012855499, -0.03332254, -0.0038227302, 0.013649216, 0.03462505, 0.031097418, 0.050065216, 0.013669568, 0.01653916, -0.029930586, -0.009653497, 0.003751499, -0.0045214724, 0.00061479135, -0.026090898, 0.017570313, 0.015955744, 0.014653234, 0.04477377, -0.003927881, 0.022766784, 0.03386525, 0.009755255, 0.04254865, -0.0062004887, 0.0124484645, 0.0062785037, 0.02199342, -0.012862283, -0.024679845, -0.01758388, -0.009619577, -0.004280644, -0.029007975, 0.0072519938, 0.04043207, 0.012882635, 0.033539623, -0.030581841, 0.03134164, 0.032209978, -0.024544166, -0.027949687, -0.0005240566, -0.0064481013, 0.009056512, -0.013167559, 0.00019864121, -0.009402492, 0.021097943, -0.039265238, 0.035710473, 0.013947708, 0.010799976, -0.03931951, -0.027610492, -0.013404995, -0.003476751, -0.030934604, 0.010861031, 0.02025674, -0.014870319, -0.028139636, -0.02223764, -0.058938563, -0.044638094, 0.0040228553, 0.021342164, -0.021599952, -0.016077854, 0.015874337, 0.015887905, -0.038722526, 0.008744453, -0.023933616, -0.029876316, 0.024720548, -0.0055899373, 0.008052494, 0.04954964, -0.033838116, -0.020283874, -0.0019452847, 0.0001994892, 0.03535771, 0.027013509, 0.013520322, -0.021355731, -0.03202003, -0.024367785, -0.00024252459, -0.006404006, 0.007516566, -0.020338146, -0.02396075, 0.03942805, 0.027461246, -0.011559773, 0.005176119, -0.021586385, 0.015209514, 0.04444814, 0.0137645425, -0.00056899997, 0.007258778, 0.0043756184, -0.0047317734, -0.008771589, -0.007340185, -0.011207011, -0.002557532, 0.0010599851, 0.031667266, -0.020731613, -0.0057866704, 0.014883887, -0.009470331, 0.04105619, -0.025439642, 0.013404995, -0.0014568435, -0.018750712, 0.014137657, 0.0020165157, 0.009165055, 0.00089971535, 0.00046045752, -0.02941501, -0.01567082, -0.003354641, -0.019578349, 0.00411783, 0.018940661, -0.014788912, 0.021233622, 0.015114539, -0.020677341, -0.012970826, -0.027162753, -0.023974318, -0.012217812, 0.042005938, 0.007821842, -0.034787863, -0.028845163, -0.0025897555, 0.00340552, -0.01548087, 0.014911022, 0.009253246, 0.03096174, -0.03003913, -0.007163803, -0.0018740536, 0.0022488644, 0.030608978, -0.026728583, 0.011139171, -0.0021521938, -0.035330575, -0.011803994, 0.0025660119, 0.012190676, -0.019279856, -0.029143654, -0.01080676, -0.013493186, 0.031124555, 0.019049205, 0.008595207, 0.0017858628, 0.017190414, -0.0025151325, 0.023499446, 0.026484363, -0.022441156, 0.040974785, -0.012373842, -0.025683863, 0.010738921, -0.053484302, 0.0023726707, 0.02291603, -0.020473825, 0.013588161, -0.017855236, -0.014761777, 0.0050845365, 0.01913061, -0.00038477458, 0.01381203, 0.0032376184, 0.03590042, -0.0051455917, 0.07880183, -0.016566295, -0.019049205, -0.0066719702, -0.0092600295, -0.013947708, -0.032671284, -0.015413031, -0.014856751, -0.027447678, 0.00255414, 0.011525854, 0.017678855, -0.0042602923, 0.039238103, -0.019049205, 0.008941187, -0.006336167, 0.03454364, -0.016430616, 0.030744655, 0.020663774, 0.011966808, -0.011593693, 0.0025456601, 0.017203981, 0.02347231, 0.0037481072, 0.017176846, 0.03183008, 0.0038634336, -0.037799917, 0.004931899, 0.015887905, -0.017923076, -0.0014466676, 0.013710271, 0.021016536, -0.018140161, 0.0069467183, -0.020134628, -0.0018621818, 0.018045187, -0.0051354156, 0.03299691, 0.026348686, 0.020270307, -0.0046435827, 0.011715803, -0.04811145, -0.015901472, 0.0020758747, 0.008025359, -0.035819016, 0.026145168, -0.056225, 0.025616024, 0.023309495, -0.002491389, 0.0068585277, -0.03690444, 0.019863272, -0.04452955, 0.0017841669, -0.008683397, -0.0601868, -0.0023591027, 0.0016866482, -0.0049217227, 0.029279333, -0.0035988614, -0.018913526, 0.009158271, 0.019836137, 0.03231852, -0.0013940923, -0.0009811221, -0.013961275, -0.0048878035, 0.003910921, -0.017393932, -0.007977872, -0.031423047, -0.009789174, -0.0008246684, 0.012977609, 0.041571766, 0.024924066, -0.035276305, 0.025154717, -0.0324542, 0.0144768525, 0.043606937, 0.07934455, 0.0069297585, -0.0149517255, 0.0016145693, 0.0120007275, -0.006027499, -0.00863591, 0.02854667, -0.0050675767, -0.038613986, 0.014693937, -0.031504452, -0.027664762, -0.037148662, -0.0153994635, 0.008690182, 0.01207535, -0.021844173, 0.0150467, -0.012984393, -0.0027237376, -0.02440849, 0.010671082, -0.0036090373, 0.021884875, 0.002052131, -0.055031035, 0.02854667, 0.023078844, -0.0061665694, 0.012577359, 0.00885978, -0.026891397, 0.0014000284, -0.0013678047, 0.012536655, 0.010277615, -0.018818552, 0.012570575, 0.020243172, 0.020541662, 0.015887905, -0.010562539, -0.027447678, -0.019591916, 0.018899959, 0.035656203, -0.008011791, 0.008873347, -0.053050134, -0.021871308, 0.0097620385, 0.03584615, 0.008642694, 0.012428113, -0.00024634053, 0.030880334, -0.0137645425, -0.02680999, 0.023933616, -0.011783642, -0.003558158, -0.011166307, -0.0162271, 0.0154265985, -0.010643946, -0.01591504, 0.018574331, 0.00054398435, -0.035059217, -0.0028797674, 0.033213995, -0.005840942, -0.007604757, -0.008296715, -0.013201478, 0.034462236, -0.033892386, -0.00934822, -0.031667266, 0.0059121726, 0.012190676, -0.022766784, 0.007692948, 0.0008624039, 0.006037675, 0.02390648, 0.0051659434, 0.0013347332, -0.005969836, -0.014191928, -0.0006347191, 0.03549339, 0.006465061, 0.005335541, 0.00017881948, 0.03139591, 0.004979386, -0.06724206, -0.014870319, -0.0024676453, 0.00953817, 0.0047453414, -0.002335359, -0.018845687, 0.037420016, 0.033403944, -0.00941606, 0.026660744, 0.011783642, 0.033919524, -0.013723839, 0.007950736, -0.009992692, 0.014748208, 0.00049480103, -0.013147207, -0.027719034, -0.0018977973, -0.009443196, -0.0023981102, 0.038613986, -0.04954964, -0.0072723455, 0.015535141, -0.0020131236, -0.028166771, 0.03975368, 0.0039244886, -0.009511034, -0.0122110285, -0.027298432, 0.0044502416, 0.015141675, -0.02081302, 0.02538537, 0.014137657, 0.02093513, -0.0128962025, -0.009456763, 0.02291603, -0.009931636, -0.014449717, -0.016620567, -0.023621555, -0.0011507198, -0.003120596, -0.045777787, 0.03264415, -0.0035344143, 0.024544166, -0.008710533, 0.03652454, 0.0038430819, 0.0045248643, 0.005193079, -0.011396959, 0.022088394, 0.04496372, -0.021844173, -0.025263261, -0.025846677, -0.032101437, 0.0072655617, -0.016742676, -0.025968786, 0.02025674, -0.00015390983, -0.008208524, 0.0070891804, 0.00089377945, -0.008120334, 0.0044264975, 0.025480345, -0.04526221, -0.032915507, 0.047405925, 0.008527368, -0.014816048, -0.03128737, -0.028872298, 0.01709544, 0.0253311, 0.009137919, -0.0025778837, 0.0097620385, 0.031043148, -0.00042420602, 0.025900947, 0.00059231964, -0.010745705, 0.012400977, -0.005474611, 0.001418684, 0.015684387, -0.022020554, -0.025181854, 0.021844173, 0.01721755, 0.008133901, -0.0022251208, 0.012048215, 0.00882586, 0.004948858, 0.0019690283, 0.00067923847, 0.013527106, -0.030419027, -0.01652559, 0.0013347332, -0.0233502, -0.0018757496, -0.004538432, -0.020487392, 0.019890409, 0.017000465, -0.01721755, 0.044828042, -0.011790426, -0.015182379, 0.0065227244, 0.020324579, 0.011424095, 0.0066346587, -0.018859254, -0.039835088, -0.007313049, -0.04230443, 0.045913465, 0.027284864, -0.0060071475, 0.0344351, -0.02644366, -0.0094974665, -0.016335642, 0.0036361727, -0.0122110285, 0.029116519, 0.017678855, 0.022644673, -0.030988876, 0.009599225, -0.013689919, 0.014856751, -0.0061428254, 0.04322704, -0.014571827, 0.033024047, -0.011145955, -0.025100447, 0.0044400655, -0.015697954, 0.04249438, -0.046157688, 0.016308507, 0.011892185, -0.008215308, 0.02117935, 0.016023582, 0.013208263, -0.04859989, -0.008303499, -0.00441293, 0.012516304, 0.039672274, 0.02321452, -0.02235975, -0.041788854, -0.012590927, 0.019619051, -0.0007059501, 0.0119735915, 0.030581841, 0.009558521, -0.027366271, 0.012828363, 0.014788912, 0.023635123, 0.021477843, 0.02434065, -0.03264415, -0.0148974545, 0.011817561, -0.026213007, 0.0050743604, 0.0073537524, -0.043525532, 0.041788854, -0.028410992, 0.04477377, 0.0023777585, 0.06262901, -0.028193908, -0.03671449, 0.030364757, 0.0005927437], "content_hash": "5484ab5b8e0d34389726fd0c5cfcfd7e"}, {"content": "Generation Choices (Beams)**: How many parallel paths to explore (1-5). With 1 beam, you get greedy decoding. More beams = more diverse results but slower. - **Number of New Tokens**: How many tokens to generate beyond your prompt (1-20). Longer generations take more time. ## How Generated Sequences Are Used\n\nAfter generation, the dashboard displays the resulting sequences. You can:\n1. **Read them**: See how the model would continue your prompt\n2. **Select one for comparison**: Click \"Select for Comparison\" to use that sequence as a baseline in ablation experiments. This lets you see how removing attention heads changes the full generated sequence, not just the immediate next token.\n\n## Greedy vs. Beam Search\n\n| Feature | Greedy (1 beam) | Beam Search (2+ beams) |\n|---------|-----------------|------------------------|\n| Speed | Fast | Slower |\n| Quality | May miss better paths | Finds better overall sequences |\n| Diversity | Only one output | Multiple alternatives |\n| Best for | Quick exploration | Thorough analysis |", "source_file": "beam_search_and_generation.md", "chunk_index": 1, "embedding": [0.0410936, -0.0025848488, 0.06470555, -0.009679434, -0.010940693, -0.025797158, -0.01522311, 0.03317406, 0.014878463, 0.05218095, 0.02418392, -0.035608586, -0.03554992, -0.0116006555, -0.0039194375, -0.055730075, -0.013932519, -0.053852852, 0.02345063, 0.06546817, 0.008315514, 0.0024125257, 0.0010192738, 0.041592237, 0.0011604321, -0.00062833825, -0.0005018457, 0.038541745, 0.04584532, -0.028026361, -0.017408315, -0.04200288, -0.0086748265, -0.022776002, -0.054820795, -0.00119618, -0.030915525, 0.0059213205, -0.001807561, -0.021324087, 0.0018607245, -0.019681515, -0.022409357, 0.00056509196, 0.009312789, 0.009760096, -0.009312789, -0.056639355, 0.0058003278, 0.054087505, 0.0012878414, 0.07890205, -0.04549334, -0.0031403163, 0.006254968, -0.012817917, -0.043909434, -0.047605217, 0.009466779, -0.016279047, 0.040330976, -0.016147057, 0.026119806, 0.047077246, 0.034347326, -0.0041064266, -0.041944213, 0.030064909, -0.022732005, 0.037603136, -0.010515385, 0.003316306, -0.00896081, -0.0048763817, -0.020708123, 0.041269585, 0.035872567, 0.004627063, -0.009386118, -0.016455038, 0.043234807, 0.0049057133, -0.0118279755, -0.037485808, 0.0011384335, -0.024755886, 0.052650254, -0.040888276, -0.0250932, -0.040770948, -0.04678393, 0.007424566, -0.044496067, 0.02067879, 0.0126785925, 0.015355102, 0.019505527, -0.03642987, -0.002529852, 0.027865037, 0.024066593, -0.016117724, -0.028261015, -0.011160681, 0.045376014, -0.01900689, -0.04115226, 0.01133667, -0.015017789, -0.032323442, -0.047751874, -0.020561464, 0.0035931233, 0.019798843, 0.022145372, 0.0015078285, -0.023142647, 0.0060129818, 0.04660794, -0.004264084, -0.05314889, -0.018537583, -0.008704158, -0.0133605525, -0.030827532, 0.010684042, 0.051946297, -0.061889715, -0.009664768, 0.013690533, 0.028524999, 0.016543033, 0.019754846, -0.022761336, -0.048338506, 0.021881387, -0.038952388, 0.015531092, -0.062828325, 0.036224548, -0.0037434478, -0.022805333, -0.010933361, 0.0026600112, -0.04478938, 0.013257892, -0.037485808, 0.017701631, -0.042530846, 0.023861272, 0.008256851, -0.031326167, -0.05188763, -0.021368084, 0.016059062, -0.021177428, -0.023289304, 0.027513059, -0.054087505, -0.012656594, -0.012641927, 0.02673577, -0.017027004, -0.011292673, -0.044672053, 0.03464064, -0.044642724, 0.036840513, -0.028290346, -0.007417233, 0.028700989, 0.0052466933, -0.00035908318, -0.005034039, 0.037955113, -0.023186645, -0.10876164, -0.06200704, -0.04364545, -0.057225987, -0.010229402, 0.018816233, -0.03807244, 0.011783978, -0.023201311, 0.0011668485, 0.0024143588, -0.0077875447, 0.017994948, -0.0144971525, -0.004590398, 0.02302532, 0.0702199, -0.026471786, -0.046490613, -0.04989308, 0.07567558, 0.038952388, -0.015428431, -0.030856863, 0.025679832, -0.036048558, 0.011849973, -0.018640244, -0.0003668744, -0.00239236, -0.019270873, -0.004084428, -0.018478919, -0.004036764, 0.021514742, -0.02229203, 0.03100352, 0.008616162, -0.04080028, 0.0058736564, 0.004872715, 0.03147283, 0.0015105783, 0.03807244, 0.002520686, -0.030739537, -0.00080111984, 0.03678185, 0.008718824, 0.032880746, -0.0015857406, -0.03009424, 0.0080221975, -0.013668534, 0.032910075, 0.016440373, -0.0043740775, -0.010119408, -0.013397217, -0.006526285, -0.012165288, -0.004949711, 0.032059457, -0.066054806, -0.022233367, -0.02372928, 0.018156271, -0.012077294, -0.02827568, 0.033995345, -0.064118914, -0.039891, -0.016601697, 0.03499262, 0.0019670518, -0.015399099, 0.0066436115, 0.010471388, 0.015589755, -0.0046417285, -0.03261676, 0.02229203, -0.030211566, -0.009474113, -0.0066179466, 0.023567956, 0.012517268, -0.0135585405, -0.058956552, 0.032528765, -0.018625578, -0.048954472, -0.06887064, -0.054322157, -0.0055583417, -0.04382144, 0.00094777794, -0.0075088944, -0.048866477, -0.04182689, 0.012209286, -0.011351337, -0.02355329, -0.024022596, -0.048103854, 0.004003766, 0.036693856, 0.0038461084, -0.0011668485, -0.015457762, 0.033760693, 0.017936284, 0.01946153, -0.021778727, 0.0023960266, -0.009452114, 0.014922461, -0.03619522, -0.005836992, 0.034054007, 0.0359019, -0.006353962, -0.03801378, 0.0012456771, 0.021852056, -0.021500077, 0.032294113, -0.02403726, -0.007516227, 0.025225192, -0.039627016, 0.044320077, 0.0038791066, 0.0076995497, 0.008777487, 0.043762773, -0.025870487, -0.064412236, -0.0053456873, -0.01704167, 0.039040383, -0.05889789, 0.040946938, -0.057607297, -0.02526919, -0.037573803, -0.001122851, 0.008476838, -0.016381709, -0.014159839, -0.016000398, -0.023567956, 0.009305456, -0.023230642, 0.013617204, -0.009180796, 0.013477879, 0.032323442, -0.050685037, -0.04528802, -0.00039826837, 0.036840513, 0.027806373, 0.03261676, -0.037867118, -0.0007809543, -0.0038057775, -0.02138275, -0.028349008, -0.006104643, -0.0076482194, 0.040477633, 0.029346284, 0.01168865, -0.0013648368, -0.03915771, -0.011446664, -0.0039414363, 0.043616116, 0.012685925, 0.010867365, -0.0273664, -0.014783136, 0.018566914, -0.018801568, 0.03801378, -0.0011705148, 0.025665166, 0.004069762, -0.032851413, -0.017760295, 0.0133605525, 0.027630385, 0.0025280188, -0.041592237, 0.025899818, 0.0017314821, 0.01974018, -0.065526836, 0.025885154, -0.014504485, 0.013030572, -0.006258634, 0.06001249, -0.031795476, 0.003241144, -0.054644804, -0.059748508, 0.00672794, 0.05042105, -0.002295199, 0.011644652, -0.049981076, -0.028378341, -0.010698708, -0.013961851, 0.019798843, 0.005386018, -0.018156271, 0.049042467, 0.003525294, 0.038541745, 0.048162516, 0.04118159, 0.0353446, -0.054234162, -0.031208841, -0.027806373, -0.018904228, 0.005408017, 0.025181195, 0.020180153, 0.05147699, 0.01984284, 0.0438801, 0.0575193, 0.04874915, -0.043792106, 0.0015866573, 0.007611555, -0.00038956056, 0.03936303, 0.022878662, 0.054996785, -0.020928109, -0.06276967, -0.057724625, -0.017202994, -0.022907995, -0.009364119, -0.0050047073, -0.006251301, 0.04437874, -0.023890603, -0.015692417, 0.059484523, 0.05329555, -0.0128765805, 0.0012264283, 0.008806818, 0.057372645, 0.03329139, -0.015281773, -0.045258686, 0.02229203, -0.007728881, -0.016733687, -0.0032484767, -0.039803006, 0.006555617, -0.061713725, -0.0176723, -0.007875539, -0.020268148, -0.03968568, 0.00934212, -0.017129665, 0.027879704, -0.03062221, -0.004418075, 0.07784612, -0.035960563, 0.061479073, 0.002390527, 0.015355102, -0.040858943, -0.015384434, 0.0013960017, 0.014409157, 0.04405609, -0.0015692416, -0.007387901, 0.02600248, 0.020722788, 0.0032686421, -0.0018222268, -0.004707725, -0.027982363, -0.009232127, -0.007758213, 0.017276322, 0.024051927, 0.0029496609, -0.020180153, -0.017862955, 0.0444374, 0.034581978, 0.05479146, -0.037221823, 0.09086935, 0.043762773, -0.014636477, 0.015941735, -0.02873032, -0.014035179, -0.029888919, 0.026134472, -0.018420257, 0.017056337, -0.03100352, -0.052708916, 0.024550565, -0.017862955, 0.0037947781, 0.02022415, 0.008535501, -0.020488136, 0.032323442, 0.032470103, -0.041416246, 0.046842594, 0.036488533, -0.0040734285, 0.0068416, -0.02428658, -0.039744344, 0.0004954294, 0.011857307, -0.0135878725, 0.029918252, 0.0052320273, 0.025445178, 0.017364318, 0.025489176, 0.026589112, -0.007985533, -0.0033126397, 0.011754646, 0.016880346, -0.061948378, -0.04147491, -0.064236246, -0.013463213, -0.013507211, 0.033144727, -0.041005604, 0.003646287, 0.025474511, -0.08195254, -0.03818977, -0.00012912786, 0.016513702, -0.023699947, -0.005664669, -0.027058419, 0.019065551, 0.009532776, 0.010280732, 0.0015059953, -0.0071899127, 0.020238817, 0.009540109, 0.0037654466, 0.0008620746, 0.018346928, -0.03698717, 0.033379383, -0.03601923, -0.028671658, 0.0040734285, -0.02694109, -0.04074162, 0.03481663, -0.020664126, -0.010361394, 0.008271516, 0.037603136, 0.0024015263, -0.025738494, 0.007160581, -0.0003441882, -0.006232969, 0.0040660957, -0.023538623, -0.020590795, -0.015105783, 0.019828174, -0.0025500176, 0.00530169, 0.003956102, 0.003411634, 0.02981559, -0.016088393, 0.014599813, 0.005609672, -0.029243624, -0.007175247, -0.004484071, -0.03510995, -0.02236536, -0.019696182, 0.02663311, 0.010625379, 0.016997673, 0.029375616, 0.010764704, 0.0020312145, 0.006702275, -0.01928554, -0.01363187, -0.02873032, -0.03027023, -0.017965617, -0.013221227, 0.027219743, -0.025049202, -0.036165886, 0.0073475703, -0.03100352, -0.0016416541, -0.028363675, -0.0032374773, 0.025562506, -0.013661201, -0.020884112, -0.00042874576, -0.0091294665, -0.017246991, -0.008777487, 0.02862766, -0.015010456, 0.0016599863, 0.03898172, -0.012957242, 0.023641285, 0.017246991, 0.026251798, 0.025899818, 0.019270873, 0.010478721, 0.0038791066, -0.03437666, 0.008821485, 0.00547768, -0.006515286, -0.0028250015, -0.012766587, 0.0015949068, -0.008176189, 0.0427655, 0.014020514, 0.02827568, -0.013059903, -0.0024491902, -0.0042970823, 0.008557499, -0.008924145, -0.035931233, -0.02100144, 0.013045237, 0.011241343, -0.006460289, -0.0014482486, -0.03044622, -0.048837144, -0.019241542, 0.06722807, -0.02330397, 0.027806373, -0.023861272, -0.008396176, -0.022174703, -0.008638161, -0.017862955, 0.004718724, 0.00706892, -0.04279483, -0.047399893, 0.010508052, 0.051770307, 0.025181195, 0.0422082, -0.010544716, -0.016631028, -0.035931233, 0.05564208, 0.007728881, -0.008168856, -0.04349879, -0.023861272, -0.00090744696, -0.026691772, -0.002861666, 0.0133605525, -0.011014023, 0.0032136454, -0.04182689, -0.029830256, 0.047194574, 0.0054850127, 0.025049202, -0.004594065, 0.04528802, 0.012641927, -0.008462172, 0.036165886, 0.011102018, 0.029859588, 0.0165577, -0.029199626, 0.0019872172, 0.04097627, -0.018214935, -0.057049997, -0.028788984, 0.021177428, 0.016235052, -0.012304614, -0.010434723, -0.0031201509, -0.03733915, 0.03733915, -0.005595006, 0.0018845565, -0.014533817, -0.020942776, -0.0095841065, 0.027879704, 0.0024766885, -0.007970868, -0.024755886, -0.040360305, 0.005690334, -0.009488778, -0.012121291, 0.015384434, -0.03329139, -0.019094884, 0.00067737704, -0.0159564, 0.008880148, 0.007391568, -0.028011696, 0.03516861, 0.02130942, 0.0046783932, -0.012216619, 0.007860874, -0.008513502, 0.02390527, -0.02918496, -0.013067236, -0.013771195, -0.013477879, 0.027777042, 0.0025243524, 0.035872567, -0.0046343957, 0.0064749545, -0.0001907701, -0.0043154145, 0.010735372, 0.00021723729, -0.025151862, -0.0051183673, 0.014262499, 0.0006155057, -0.02330397, 0.0541755, 0.006922262, 0.029786259, -0.011622653, 0.010060745, 0.04223753, 0.011131349, -0.00054950954, -0.033496708, 0.028481001, -0.0004035389, -0.007369569, -0.016675025, -0.008161522, -0.004744389, 0.011380668, 0.0037086166, 0.014416491, 0.018962892, -0.004627063, 0.05188763, -0.035080615, 0.052122287, -0.014225835, -0.0496291, -0.005855324, 0.007142249, 0.013067236, -0.029947583, 0.0042604175, -0.025342518, 0.010801368, 0.00067554385, -0.02309865, -0.0008794902, -0.03437666, 0.0029734927, -0.014555816, 0.028495668, -0.003347471, -0.02100144, 0.009716098, 0.0010641877, -0.12647794, -0.022204036, 0.004971709, -0.0122239515, -0.0021173763, -0.02739573, 0.04291216, 0.015648419, -0.013917853, 0.0159564, -0.025474511, -0.031062184, 0.029947583, -0.0017424815, 0.021808058, -0.04200288, 0.0020000497, -0.018640244, 0.014907795, -0.02617847, -0.024799883, 0.010852699, -0.009422783, -0.013382551, -0.025870487, -0.000774538, 0.003145816, 0.009298123, 0.0013923353, -0.024315912, -0.015824407, 0.056756683, -0.025811825, 0.04115226, 0.0061339745, -0.008366844, -0.009217461, -0.02120676, 0.015516426, 0.004194421, -0.007123917, -0.023861272, 0.022658676, -0.0052430267, 0.0049790423, -0.0059873164, 0.00888748, 0.0035656248, 0.026838431, 0.023597287, -0.005972651, -0.004594065, -0.006973592, 0.00017770835, 0.022204036, -0.015017789, 0.0033438045, 0.015883071, 0.028583663, 0.010112075, 0.021250758, 0.03062221, 0.044818714, -0.033203393, 0.01141, -0.034669973, 0.024755886, 0.008689492, -0.016851015, 0.007765546, -0.016792351, 0.011373335, -0.016059062, 0.015325771, -0.007890205, 0.0124512715, 0.04549334, 0.029888919, -0.013529209, -0.01991617, 0.019696182, -0.039597686, -0.029302286, 0.034405988, 0.0061743055, 0.020004164, -0.017540308, -0.026207801, 0.00091615476, 0.029170295, 0.023245307, -0.027953032, -0.0004638062, 0.007065254, -0.059132542, -0.015941735, -0.0038131103, 0.013881189, 0.027014421, 0.004590398, 0.04534668, 0.02782104, -0.014152505, -0.039245706, 0.0052466933, 0.014599813, -0.0010806868, 0.0014289998, -0.017202994, -0.006672943, 0.017686965, 0.01949086, 0.0016746521, 0.008916812, -0.016117724, 0.029170295, 0.04531735, -0.0008354928, 0.0055876733, -0.011307339, 0.0025756827, -0.005122034, -0.00078828726, 0.014885796, 0.0122826155, 0.013089235, 0.012539267, -0.009613438, 0.010676709, 0.00092807075, -0.005276025, 0.016807018, 0.020532133, -0.001030273, -0.0048763817, -0.00059992325, 0.0025023536, -0.027043752, -0.070102565, 0.0047957194, -0.024887878, -0.0039267703, 0.053324882, -0.0068709315, -0.019270873, -0.03464064, -0.025166528, 0.0037837788, -0.012832583, 0.016748354, 0.01855225, -0.019813508, 0.0008749072, 0.016352378, -0.017877622, 0.023171978, 0.00019054094, -0.017056337, 0.0043007485, 0.018420257, 0.038512416, 0.021192094, 0.0541755, 0.007868207, 0.013221227, -0.0036169551, -0.0069442606, 3.242519e-05, 0.0071385824, 0.0038791066, -0.029551605, -0.017202994, 0.026603777, 0.027278405, 0.055143442, 0.0037874451, 0.017789626, 0.036693856, 0.007516227, 0.0507437, 0.012121291, 0.045786656, -0.0066252793, 0.030974189, 0.006185305, -0.016161721, -0.012656594, 0.005910321, 0.02113343, -0.01868424, -0.0070322556, 0.045962647, 0.007230244, 0.02953694, -0.003517961, 0.014783136, 0.029551605, -0.0199455, -0.03062221, -0.009386118, 0.017246991, 0.021338752, -0.0065372842, 0.011659319, 0.0117986435, 0.030827532, 0.0006530868, 0.035637915, 0.023010654, 0.04346946, -0.04546401, -0.02438924, -0.014482487, 0.0051696976, -0.03862974, 0.03173681, 0.010302731, -0.007028589, -0.025342518, 0.0012355944, -0.06288699, -0.030123573, -0.019329537, 0.003598623, -0.017173663, -0.024051927, 0.022614678, 0.020077493, -0.045757324, 0.012502602, -0.004707725, -0.018258933, 0.023318637, -0.01913888, -0.00650062, 0.027879704, -0.023978598, 0.012839916, -0.0038204433, -0.00073145726, 0.048133187, 0.001511495, 0.00199455, -0.008080861, -0.026618443, -0.028847646, -0.00071220833, -0.011930636, 0.011886639, -0.02113343, -0.013426548, 0.024447903, 0.026339794, -0.020180153, 0.013807859, -0.017408315, 0.0036444536, 0.022262698, 0.034669973, 0.0075895563, 0.0013547541, 0.030680872, 0.008667493, -0.024169253, 0.006493287, -0.014101176, -0.0068782647, 0.0046380623, 0.037984446, -0.029170295, 0.003550959, 0.025386516, -0.0030761536, 0.04382144, -0.010772036, 0.0077362144, 0.00014665809, -0.016997673, 0.02236536, -0.021984048, 0.0012355944, -0.011050687, -0.008447506, -0.023113316, -0.03153149, -0.010706041, 0.00027315068, 0.013939852, 0.027659716, 0.002350196, 0.018346928, 0.032880746, 0.0012291782, 0.0032448103, -0.020048162, -0.015883071, 0.0019835506, 0.024653226, 0.026985088, -0.026662441, -0.015267108, 0.0050890357, 0.009481446, -0.022893328, 0.013719864, 0.017554974, 0.039128378, -0.04150424, -0.015369768, -0.0017608138, -0.008183521, 0.053207554, -0.0124512715, 0.009481446, -0.0001843538, -0.0116006555, -0.012062628, -0.006493287, 0.00055730075, -0.008462172, -0.015868405, -0.01282525, -0.015765745, 0.033027403, 0.0012896745, -0.030020911, 0.00024152754, 0.0020385475, -0.0056573357, 0.025078533, 0.026339794, -0.023142647, 0.032000795, -0.012047962, -0.013485212, 0.022424022, -0.0467546, 0.0039267703, 0.006518952, -0.011651985, 0.008058862, -0.0017204827, -0.012568599, -0.006401626, 0.0071202503, 0.019637518, 0.0155457575, -0.010948027, 0.015311105, 0.004553734, 0.08746689, -0.014035179, -0.020664126, 0.002927662, -0.011952635, -0.0061669727, -0.026574446, -0.0056610024, -0.012187287, -0.01009741, -0.004480405, 0.021177428, 0.031150179, 0.0018772236, 0.02830501, -0.0093274545, 0.008850816, -0.015384434, 0.034523316, -0.008132191, 0.025063869, 0.04511203, -0.0032393106, 0.012407275, -0.0032741418, 0.024403907, 0.026266465, 0.004605064, 0.0066399453, 0.033056732, 0.002286033, -0.023494625, -0.010794035, 0.0009771095, -0.02012149, -0.0065666162, 0.0025555172, 0.031766143, -0.01718833, 0.004007432, -0.009312789, 0.00593232, 0.014438489, 0.0060753115, 0.022130705, 0.00020027996, 0.018508252, 0.019828174, 0.010390726, -0.032059457, -0.018640244, -0.003088986, -0.010060745, -0.019080218, 0.0433228, -0.07033722, 0.03208879, 0.024301246, 0.008594165, 0.0018057277, -0.04291216, 0.013140565, -0.03772046, 0.010896696, -0.0023465294, -0.08113126, -0.002748006, 0.0006081728, 0.0071459156, 0.044173416, -0.012341279, -0.016000398, 0.00883615, 0.019798843, 0.019784177, 0.009562108, 0.0060569793, -0.019886836, -0.0016966508, 0.01133667, -0.0072045787, -0.028935641, -0.035461925, -0.036107223, -0.021016104, 0.022482686, 0.04625596, 0.026017146, -0.026985088, 0.038952388, -0.03783779, 0.015516426, 0.04200288, 0.07778745, 0.0088728145, -0.01701234, -0.013609871, -0.0111166835, -0.0072485763, -0.0025426846, 0.032734085, 0.004480405, -0.020517467, 0.008484171, -0.04402676, -0.031326167, -0.029170295, -0.006493287, 0.019872172, 0.0029606603, -0.0072265775, 0.021192094, 0.015663084, 0.0098114265, -0.02418392, 0.02512253, 0.0039451024, 0.020136155, 0.0131039005, -0.044877376, 0.03437666, 0.024975874, -0.005650003, 0.004058763, 0.00087261567, -0.024858547, 0.0023483627, 0.009444781, 0.014753804, 0.0062733, -0.0052246945, 0.005631671, 0.014526484, 0.03173681, 0.034493983, -0.0017543974, -0.018185603, -0.018669575, 0.0070725866, 0.036869843, -0.020253483, 0.020722788, -0.04241352, -0.017408315, 0.010024081, 0.02257068, 0.014284498, -0.007457564, -0.011930636, 0.010478721, -0.013030572, -0.01855225, 0.014394492, -0.010750039, 0.015032454, -0.025738494, 0.00273884, 0.012957242, 0.007402567, -0.021338752, 0.030710204, 0.010852699, -0.02526919, -0.0021283755, 0.028363675, 0.006526285, -0.011351337, -0.017071001, -0.010647377, 0.032352775, -0.007948869, -0.018478919, -0.011959967, 0.008066195, 0.0041650897, -0.0135585405, -0.011439331, 0.0047370563, 0.010082744, 0.026559781, -0.0020147155, 0.011102018, 0.008572166, -0.013514543, -0.010764704, 0.0133605525, -0.002295199, -0.0017663134, 0.0035802906, 0.03572591, 0.0055876733, -0.061127093, -0.0026545115, -0.0013905021, 0.02236536, -0.012517268, -0.025943816, -0.010368727, 0.025958482, 0.023391966, -0.012069961, 0.017452313, -0.0016040729, 0.03769113, -0.03390735, 0.0039414363, -0.007109251, 0.02648645, -0.0033511375, -0.011549325, -0.016719023, -0.0030999854, -0.020312145, -0.017349653, 0.04244285, -0.04698925, 0.0021467078, 0.008821485, -0.016205719, -0.03437666, 0.014071844, -0.00067004416, 0.0073989006, 0.00031966882, -0.037925784, 0.0035124612, -0.00049497106, 0.0047700545, 0.024975874, 0.008630829, 0.0064492896, -0.014944459, -0.020605462, 0.01105802, -0.023142647, -0.0064162915, -0.03282208, -0.008491503, 0.0032264781, 0.019109549, -0.036107223, 0.011527326, 0.0021962048, 0.01739365, -0.027307736, 0.050127737, -0.007178914, -0.017598972, -0.0059433193, 0.004854383, 0.017100334, 0.020957442, -0.031766143, -0.020517467, -0.012113959, -0.03769113, -0.005496012, -0.005895655, -0.02691176, 0.01984284, -0.010691375, 0.002163207, 0.0046563945, -0.00021242508, -0.016088393, 0.0033896351, 0.012187287, -0.02145608, -0.012590597, 0.043381464, -0.010508052, 0.0017809792, -0.044173416, -0.025386516, 0.024579896, 0.02554784, -0.008491503, 0.010647377, -0.005968984, 0.019593522, 0.018141607, 0.011813309, -0.0055876733, -0.019901503, 0.017422982, 0.013015905, 0.004458406, 0.0119966315, -0.013675867, -0.018244267, 0.020341476, 0.032294113, 0.011717982, -0.007692217, -0.005122034, -0.01209196, 0.0059066545, -0.0091001345, -0.0034996287, 0.025987813, -0.03390735, -0.016381709, 0.012751921, -0.014225835, 0.0042970823, 0.010060745, -0.020444138, 0.034963287, 0.01966685, -0.0037727794, 0.035931233, -0.008117526, -0.012590597, 0.006485954, 0.032206118, 0.0205468, -0.004304415, 0.010207403, -0.037045833, 0.0015912403, -0.053970177, 0.034288663, 0.021045437, -0.0211041, 0.01855225, -0.017525641, -0.0033034736, -0.014951793, 0.019534858, -0.013551207, 0.03493396, 0.019270873, 0.007875539, -0.020414807, -0.00070258393, -0.022556014, 0.0115419915, -0.014724473, 0.042589508, -0.007838875, 0.016117724, -0.00752356, -0.022174703, -0.00951811, -0.02330397, 0.0393337, -0.04135758, 0.013353219, 0.003655453, -0.013903187, 0.019373534, 0.025841156, 0.015237776, -0.024931876, -0.0014180004, -0.00096611015, 0.009804093, 0.024755886, 0.020414807, -0.015252441, -0.037456475, -0.014225835, 0.035432596, 0.0005687584, 0.0063906265, 0.022805333, -0.006705941, -0.03238211, 0.0025445179, 0.013213894, 0.014907795, 0.037045833, 0.029551605, -0.028994305, -0.0051330333, 0.008198188, -0.021602737, -0.0022163703, 0.02229203, -0.01830293, 0.023259973, -0.030886194, 0.023010654, -0.00067004416, 0.044877376, -0.033232722, -0.0290823, 0.021294754, 0.004553734], "content_hash": "a391a36b69807fd98403083dcc620382"}, {"content": "# Dashboard Overview\n\n## What Is This Dashboard?\n\nThe Transformer Explanation Dashboard is an interactive web application that lets you look inside transformer-based language models as they process text. Instead of treating the model as a \"black box,\" you can see exactly what happens at each step when the model makes a prediction.\n\n## How to Navigate\n\nThe dashboard has several main sections, from top to bottom:\n\n### 1. Header and Sidebar\nAt the top is the dashboard title. On the left edge, there's a collapsible **sidebar** (click the hamburger menu icon). The sidebar contains advanced configuration options for selecting which internal model components to hook into. Most users won't need to change these settings.\n\n### 2. Generator Section\nThis is where you start. It contains:\n- **Model dropdown**: Choose which transformer model to load (e.g., GPT-2, Qwen2.5)\n- **Prompt input**: Type the text you want the model to analyze\n- **Generation settings**: Control beam search parameters (number of beams and tokens to generate)\n- **Analyze button**: Click to run the model and see results\n\n### 3. Generation Results\nAfter clicking Analyze, you'll see one or more **generated sequences** -- these are the model's continuations of your prompt. If you used beam search with multiple beams, you'll see multiple possible continuations. Click \"Select for Comparison\" on any sequence to use it in ablation experiments.\n\n### 4. Pipeline Visualization\nThis is the core educational section. It shows **5 expandable stages** that your text passes through:\n1. **Tokenization**: How your text is split into tokens\n2. **Embedding**: How tokens become number vectors\n3. **Attention**: How the model finds relationships between tokens\n4. **MLP**: How stored knowledge is retrieved\n5. **Output**: What the model predicts and how confident it is\n\nClick any stage to expand it and see detailed explanations and visualizations.", "source_file": "dashboard_overview.md", "chunk_index": 0, "embedding": [-0.0074431063, 0.029747594, 0.009833095, -0.010689766, -0.00012260329, -0.004956898, -0.01614018, 0.04390129, -0.017878354, 0.063666806, 0.011316749, -0.039580688, -0.017965263, -0.035409074, 0.0029347194, -0.008386686, -0.008368063, 0.032081712, 0.017381733, 0.097536355, 0.03558289, -0.038115658, 0.020137979, 0.0474273, -0.011329166, -0.0052300394, -0.011937526, 0.041865144, -0.019293724, -0.0070458096, 0.0078093642, -0.024483413, -0.033745393, -0.00080157723, 0.0028276355, -0.027314153, 0.003774319, 0.038786095, -0.011745085, 0.03084016, 0.0070458096, 0.002866434, -0.026594052, 0.06401444, -0.035359412, 0.04337984, -0.022534175, -0.021553349, -0.03819015, 0.029449621, 0.0072072116, 0.034937285, -0.032603163, -0.0044416534, -0.013123209, -0.008926761, -0.06356748, -0.008703282, -0.026842361, -0.033074956, 0.00640641, 0.005993594, 0.0048606773, 0.07434416, -0.010242807, -0.012651418, -0.023862636, 0.012055473, -0.035682216, -0.009764809, 0.044745546, -0.0028881612, -0.003805358, -0.0018157703, 0.005503181, -0.020733925, 0.00572666, 0.040648423, -0.04320602, -0.039481364, 0.002866434, -0.039531026, -0.022608668, -0.018312898, -0.0043299138, 0.00464651, -0.019318555, 0.018573623, 0.0017521407, -0.06912964, -0.015804961, -0.0022006514, -0.046086423, 0.020609768, 0.037395556, 0.021503687, -0.045962267, -0.007890065, 0.0017521407, 0.049835913, 0.049314458, -0.037321065, 0.02041112, -0.017158253, 0.037519712, -0.037097584, 0.013408765, -0.028307393, -0.0455898, -0.014786889, -0.08492218, -0.02773628, 0.0036780988, 0.0076231314, 0.022310697, 0.015345587, 0.015829792, -0.010801505, -0.011298127, -0.043181192, -0.034515154, 0.0007845059, 0.010609065, 0.006431241, -0.036104344, 0.024309594, 0.010180729, -0.012620379, -0.004252317, 0.025575979, -4.665521e-05, 0.006431241, -0.0065057343, -0.0488675, -0.02753763, 0.0038426046, -0.045142844, 0.0219134, -0.021863736, 0.030393202, 0.043007374, -0.001118949, -0.009944835, -0.00091176503, -0.06530565, -0.025973275, -0.011012569, 0.03081533, -0.05408202, 0.018921258, 0.012192044, -0.007480353, -0.030442864, -0.047129326, 0.01885918, -0.018772272, 0.013892971, 0.028183239, 0.0010235048, -0.05189689, 0.008194245, 0.034713805, -0.042187948, -0.011322957, -0.06192863, 0.016537478, -0.029822087, -0.005580778, -0.053088777, -0.026867192, -0.014041957, -0.024272349, 0.005965659, -0.0016698879, 0.009864134, -0.011006362, -0.067043826, -0.07657895, 0.0018437052, -0.041914806, -0.019380633, 0.010658727, -0.029052325, -0.0026724415, -0.03203205, -0.008119753, -0.020448368, 0.029350298, -0.009423383, 0.0028710898, -0.018561207, 0.048395712, 0.024595153, -0.0038301891, -0.0383888, 0.01026143, 0.057955664, 0.023043212, 0.036228497, 0.013284611, 0.04760112, 0.02582429, 0.029126817, -0.026469896, -0.010354547, 0.06942761, -0.014054373, 0.020249719, -0.01744381, -0.040449776, 0.03516076, -0.025898783, 0.021292623, 0.013979879, -0.03647681, -0.0206346, -0.04171616, 0.03195756, 0.03081533, 0.023875052, -0.019877253, 0.035483565, 0.007983182, -0.00918128, -0.008703282, 0.053039115, 0.012353446, -0.026867192, -0.0037929425, 0.043727472, 0.03203205, -0.004109538, -0.016040856, 0.010342131, 0.024247518, -0.07737354, -0.0040226295, -0.016425738, 0.022025138, 0.0006328037, -0.006636097, 0.016624385, 0.004143681, 0.018077003, 0.04623541, 0.022559006, -0.047030002, -0.0669445, 0.033819888, 0.052989453, 0.04171616, 0.022919057, -0.033397757, -0.0030914654, 0.002579325, 0.015581482, -0.04037528, 0.010571818, -0.020324212, 0.0005532668, 0.020423535, 0.005341779, -0.053237766, -0.021652672, -0.011124309, -0.0051462343, 0.0028245316, -0.028034251, -0.051151957, -0.044199266, 0.0029455828, -0.027636955, -0.0040443568, 0.03868677, -0.055919517, -0.03225553, 0.035011776, -0.005922205, 0.009212319, 0.017990094, -0.011614722, -0.03764387, 0.03729623, 0.031361613, -0.016165012, 0.009684108, 0.02701618, -0.008324608, 0.0102365995, -0.024570322, 0.026767869, 0.0038922667, 0.042014133, -0.02704101, -0.020435952, 0.018735025, 0.016810618, 0.021093974, 0.014029542, -0.006716798, 0.011353997, -0.048768178, 0.015047614, 0.01364466, 0.018672947, 0.01705893, -0.010745635, 0.0017924912, 0.012136174, -0.012061681, -0.034912452, -0.019169569, 0.026544388, -0.016227089, 0.015705638, -0.008343232, 0.041219536, 0.003985383, 0.026718207, -0.03647681, -0.027388645, -0.026743038, -0.015904285, -0.00995725, -0.019529618, 0.02074634, -0.012682457, 0.0041778237, -0.013272194, -0.004305083, 0.06451106, -0.021975476, 0.021205714, 0.011273296, -0.044671055, -0.033770222, 0.023316354, 0.03913373, -0.059197217, 0.042560413, -0.06570295, -0.01506003, -0.07528774, -0.04489453, -0.027810773, 0.03156026, -0.023887467, 0.03277698, 0.027761111, -0.0021773723, -0.027487969, -0.04228727, -0.01636366, -0.023949545, 0.03317428, 0.04534149, -0.042113457, 0.00884606, 0.010180729, -0.014501331, -0.01836256, 0.010801505, 0.02820807, 0.013930217, -0.034912452, -0.0061146454, -0.034068197, 0.009144033, 0.020137979, -0.022198956, 0.012787989, 0.007939727, 0.02915165, 0.004680652, -0.05711141, -0.040474605, 0.00865362, -0.028580535, -0.04933929, 0.04489453, 0.03488762, 0.0053541944, -0.044671055, -0.015953947, 0.0282329, 0.050407026, 0.0011492119, -0.0077472865, -0.036104344, 0.04745213, 0.017332071, -0.055373237, -0.0030418034, -0.029275805, -0.007629339, 0.029871749, 0.02584912, 0.0032528671, 0.05904823, 0.00089934946, 0.012949391, 0.0058570234, -0.0611837, -0.035185594, -0.0057142447, 0.011409866, 0.016748542, 0.01955445, 0.01542008, -0.024595153, 0.05477729, -0.03503661, 0.0066857594, -0.02481863, 0.008504634, 0.009454421, 0.01638849, 0.037097584, 0.034266844, 0.04077258, -0.03602985, -0.030020736, -0.043280516, 0.0009482356, -0.050009727, 0.047253482, 0.029002663, -0.027910097, -0.0051679616, -0.033074956, 0.02041112, 0.033074956, 0.04534149, 0.0013137176, 0.01314804, -0.061829306, 0.057260394, 0.019678604, 0.0026693377, 0.002917648, 0.020932572, -0.036923766, -0.004395095, -0.027785942, -0.045142844, 0.034515154, -0.012974222, -0.067639776, -0.026892023, -0.04648372, 0.02560081, 0.04313153, -0.0029905892, 0.00942959, -0.05293979, -0.030542187, 0.03846329, 0.012074097, 0.06664653, 0.021454025, 0.026072599, 0.04293288, -0.022136878, 0.0074927686, -0.02121813, -0.0018995751, 0.0003798374, -0.030169722, 0.014128866, -0.00718238, -0.021230545, -0.015817378, 0.005099676, -0.01245277, -0.015457327, -0.010404209, 0.03508627, -0.032603163, -0.00037091377, -0.04628507, -0.027388645, -0.0025622537, -0.022062385, 0.016674047, -0.004190239, 0.10548229, -0.058501948, -0.007120303, 0.005965659, -0.01469998, -0.0051772734, -0.020920157, 0.018077003, -0.012595548, 0.022894226, -0.007486561, -0.06252458, -0.013259779, -0.02099465, -0.003982279, 0.015109692, 0.020349042, -0.0136074135, 0.07459246, 0.031609923, 3.157229e-05, 0.019244062, 0.031932727, 0.013979879, 0.011602307, -0.052095536, -0.03602985, 0.00918128, 0.009702732, -0.011236049, 0.013359103, 0.013421181, 0.0047023795, 0.035135932, 0.013023884, 0.0050251833, -0.0156187285, -0.004196447, -0.023440508, -0.0002630539, -0.018623285, -0.053138442, -0.063368835, -0.045316663, -0.002138574, 0.0018902634, -0.028481212, -0.00027876731, 0.02468206, -0.076231316, -0.026966518, 0.018424638, -0.00829357, -0.03652647, -0.008231492, 0.018983336, 0.040275957, 0.019206814, -0.0050438065, 0.012620379, 0.01730724, -0.0051927925, -0.016736126, 0.01120501, 0.011099478, 0.025749795, -0.04645889, 0.030293878, -0.0227949, -0.0035880862, 0.010224184, -0.0039729676, -0.022372773, 0.002992141, -0.045838114, -0.0038426046, 0.014414423, 0.01827565, -0.014327514, -0.014128866, 0.020311797, -0.00051990006, -0.020709094, -0.026668545, -0.0044944193, 0.012825236, -0.00912541, -0.0016217778, 0.032876305, 0.0072506657, -0.011763709, -0.003755696, 0.01825082, -0.020758756, 0.01400471, -0.027214827, -0.045242168, -0.0211933, 0.021975476, -0.03342259, 0.040226296, -0.001223705, 0.011087063, -0.014836551, 0.00033696508, 0.029548947, 0.0010467839, -0.016798204, 0.016016025, -0.036104344, -0.02560081, 0.021665089, -0.010168314, -0.036923766, 0.0018173222, 0.0025995004, -0.040126972, -0.026345741, -0.012912145, 0.0075362227, -0.019231645, -0.011385035, 0.015295925, -0.008610166, -0.002791941, -0.018064586, -0.021776829, -0.03503661, -0.007008563, -0.010546987, 0.03908407, 0.002998349, 0.018598454, 0.04477038, -0.0126017565, -0.008020428, -0.03625333, 0.022621084, -0.037097584, 0.007939727, 0.0046837563, -0.005602505, -0.009243357, -0.04151751, 0.021503687, -0.037668698, 0.022360358, 0.053933036, -0.010391793, -0.0054007526, 0.057806678, 0.005763907, -0.015854623, 0.009851718, -0.02241002, 0.0038239812, -0.019417878, -0.023390846, -0.0022766967, 0.00776591, 0.018027341, 0.0016869593, 0.030293878, -0.038314305, -0.008976423, -0.049612433, -0.011993396, 0.06024012, 0.0068657845, 0.024595153, -0.002394644, 0.019703437, -0.016090518, -0.009342682, 0.008194245, -0.044944197, 0.017567966, -0.005940828, -0.027115503, -0.03560772, 0.04695551, 0.017468642, 0.025749795, 0.027463138, 0.012458978, -0.051598914, 0.021230545, -0.002998349, 0.019542035, 0.0050686374, -0.019157153, 0.022248618, -0.042634908, 0.006493319, 0.04936412, -0.01656231, -0.00385502, -0.03156026, 0.0282329, 0.03764387, 0.013334272, 0.007300328, -0.0052517666, 0.034763467, 0.017481057, -0.021441609, 0.008672243, 0.00068712165, -0.012595548, -0.02146644, -0.017071344, 0.001745933, 0.05641614, 0.010081405, -0.10786607, -0.014290268, -0.020212471, -0.022919057, -0.020510444, -0.011211217, -0.028034251, -0.005754595, 0.029772425, 0.028580535, -0.04626024, -0.028034251, 0.002841603, -0.0012462082, -0.017158253, -0.02421027, 0.03200722, -0.0113788275, 0.018002508, 0.00890193, -0.009951042, -0.01789077, 0.019852422, -0.038512953, -0.014327514, 0.005915997, -0.06361714, -0.004385784, -0.00069682125, -0.024284763, -0.0020687366, 0.016078103, 0.007368613, -0.018424638, -0.051847227, 0.028034251, 0.0035446319, 0.0010972219, 0.0027795252, -0.02773628, -0.021131221, -0.032578334, 0.025327668, -0.012074097, 0.022136878, -0.013222532, -0.013992295, -0.013210117, 0.034242015, -0.015842209, -0.026767869, 0.016065687, 0.009547537, -0.001745933, 0.00057189004, -0.004143681, 0.014389591, 0.049537938, 0.0011057575, 0.01944271, -8.905471e-06, 0.016798204, 0.0011065336, -0.018809518, 0.029524116, -0.007914896, -0.018735025, 0.021975476, 0.017741783, 0.016810618, -0.011142933, 0.02488071, -0.007840403, 0.026494727, -0.010056574, 0.04176582, -0.028282562, 0.008442556, 0.003594294, -0.01614018, 0.0044944193, -0.03178374, 0.013594998, -0.0027407268, -0.010180729, 0.009634446, -0.0052052084, -0.02074634, -0.011087063, -0.016736126, -0.01838739, -0.037047923, 0.01758038, 0.03893508, -0.010410417, -0.004584432, -0.018077003, 0.014575825, -0.10026777, -0.051847227, -0.0041033304, -0.008119753, 0.020001408, -0.041840315, 0.0052300394, -0.009417174, 0.008399102, -0.032901138, -0.013930217, -0.01656231, 0.021019481, 0.016599555, 0.02939996, -0.059594512, 0.03319911, -0.008337024, 0.036054682, -0.0022518656, -0.022633499, -0.0073313667, 0.0144516695, 0.01314804, -0.059693836, -0.008666036, -0.00771004, 0.021168469, 0.021652672, 0.010217976, -0.008802606, 0.02704101, -0.014575825, 0.0019119906, 0.004019526, 0.0048824046, -0.0056863097, -0.026221586, 0.014563409, -0.0030837057, -0.0035539437, -0.024173025, 0.023452925, -0.024135778, 0.019864837, 0.022049969, -0.016301582, -0.019455126, 0.003541528, 0.015718052, -0.0105904415, 0.019157153, -0.00029157082, 0.010658727, 0.017828692, 0.002109087, 0.0043081865, -0.009920003, 0.030467695, 0.0037153454, 0.01339635, 0.031907894, 0.016450569, 0.0040505645, 0.006896823, 0.024247518, 0.014414423, 0.016996851, -0.047625948, -0.016897528, -0.010702182, 0.00918128, -0.017567966, -0.0275128, -0.005217624, 0.028903339, 0.026171923, 0.013446012, 0.011142933, -0.020324212, 0.022174126, -0.013272194, -0.055869855, 0.030666344, 0.0046558212, 0.03173408, -0.012787989, -0.015395249, 0.009491668, 0.03242935, 0.0122479135, 0.0065181497, 0.0032808022, -0.024843462, -0.0184743, -0.003333568, -0.00021571972, 0.026494727, 0.044199266, -0.008808814, 0.03771836, 0.013843309, -0.044447575, -0.016102934, -0.028506042, 0.026544388, -0.020733925, -0.016512645, -0.05433033, -0.0017273097, -0.0067354217, 0.0067912913, 0.016065687, -0.00044501893, -0.0126576265, 0.03846329, 0.030368371, 0.032454178, 0.019852422, -0.022844564, 0.017605212, 0.010993946, -0.044050276, -0.0033118408, -0.005760803, 0.01888401, 0.01014969, -0.004100227, -0.0042305896, 0.01614018, 0.004823431, -0.005866335, 0.007815572, -0.0156187285, -0.009876549, 0.010938076, 0.00077480625, 0.024756555, -0.007834195, 0.004562705, -0.03913373, 0.031634755, 0.038984742, -0.006977524, -0.016326413, -0.0082377, -0.004820327, -0.035533227, -0.013768815, 0.0021059832, 0.042113457, 0.029375128, -0.020684263, 0.033770222, 0.01777903, 0.020137979, -0.021007067, -0.07186105, -0.03461448, 0.016152596, 0.024024038, 0.020696677, 0.016177427, -0.02562564, 0.011831994, -0.01350809, -0.03103881, 0.01627675, -0.034068197, 0.015705638, -0.011825786, 0.02560081, 0.01730724, -0.005673894, 0.014501331, 0.01921923, 0.011186386, 0.058005325, 0.017183084, 0.03749488, -0.031833403, 0.050407026, -0.0001377347, 0.0026274351, 0.015407665, -0.00879019, 0.0027205516, -0.0021975476, -0.02773628, -0.0003332792, 0.0029067844, 0.009559953, -0.010621481, 0.031162964, -0.038115658, -8.6714674e-05, 0.039059237, 0.0019337178, -0.033596408, -0.013706738, -0.018946089, 0.041592006, 0.022943888, -0.01730724, 0.0006747061, 0.031088471, -0.00848601, 0.01566839, 0.014513747, 0.010478701, -0.011583683, 0.0045782244, 0.0006448313, 0.013818477, -0.005388337, 0.0009994496, 0.03607951, -0.0041064345, 0.0069278623, -0.015879454, -0.039754506, -0.055174585, 0.007530015, -0.00015558203, -0.028977832, -0.0041405773, 0.0065802275, 0.019157153, -0.008684658, 0.010683558, 0.0011468839, -0.030244214, 0.026171923, 0.0014277853, 0.010391793, 0.031088471, 0.015358003, 0.004395095, 0.010602857, 0.007921104, 0.025948444, 0.007666586, -0.004066084, -0.006040152, -0.0111056855, 0.015792547, 0.036228497, 0.030964315, 0.0064746956, -0.02609743, -0.008802606, 0.039829, 0.025650471, -0.0054814536, 0.014104035, -0.006859577, 0.026221586, 0.033993702, -0.015233847, -6.741241e-05, 0.0037215531, 0.037668698, 0.00942959, -0.024731724, -0.0009559953, -0.0007507512, -0.02249693, -0.0063226055, 0.03463931, 0.021156052, -0.0240737, 0.028630197, -0.008554296, 0.014253021, -0.013992295, -0.025501486, 0.0060587754, -0.008274946, 0.0104538705, -0.0026243313, -0.0024179232, -0.021888567, 0.023030797, -0.028183239, 0.008641205, -0.018834349, -0.03558289, 0.007945935, 0.0015604761, -0.022571422, -0.03456482, 0.014526162, -0.0052734935, -0.031634755, 0.0024846566, 0.010329716, -0.015258678, 0.011316749, -0.004714795, -0.028530873, 0.0020004513, 0.015966363, -0.019852422, -0.05616783, 0.00782178, -0.044447575, 0.027239658, -0.049413785, -0.01946754, 0.018263236, -0.032578334, -0.0029595504, -0.0008652068, 0.05095331, 0.027562462, 0.0029579985, -0.0042895633, -0.0060153212, 0.018399807, -0.011720254, -0.004559601, 0.019926915, -0.0030169722, 0.014613071, -0.023068042, -0.016785787, 0.019393047, -0.0014696876, 0.008510841, 0.014079204, 0.0007782981, 0.0063195014, -0.00094978756, -0.004671341, -0.015581482, -0.028307393, -0.047526624, -0.028133577, 0.0069278623, -0.01849913, 0.01270108, 0.012266537, -0.02110639, -0.01564356, 0.018623285, -0.0034949698, 0.026767869, -0.021863736, 0.033795055, 0.007567262, 0.059147555, -0.033521913, -0.007523807, 0.012309992, -0.004115746, -0.018958505, -0.030244214, 0.01586704, -0.020796003, 0.0049879365, -0.004094019, 0.0143150985, -0.015469742, -0.001978724, 0.024657229, -0.007827988, -0.0020128668, -0.021391947, 0.032081712, 0.00057344197, 0.027959758, 0.013495674, -0.009876549, -0.00065996265, 0.0063939947, 0.003858124, -0.000938148, -0.017990094, 0.038016334, 0.025650471, 0.003858124, -0.041194707, 0.021367116, 0.010460079, -0.011552645, 0.025501486, 0.013930217, 0.0088770995, -0.04360332, -0.0164754, 0.020920157, -0.0052269353, 0.0018700883, -0.014911043, -0.0070458096, 0.03503661, -0.006617474, -0.021516101, -0.030492526, -0.028754352, -0.005813569, -0.0034639311, 0.010410417, -0.015209016, 0.035458736, -0.06192863, 0.004246109, 0.0069216546, 0.0022254826, 0.006282255, -0.026196755, 0.03968001, -0.011807163, 0.026668545, 0.024446165, -0.04504352, -0.012912145, 0.01755555, -0.021031898, 0.023837805, 0.014873797, 0.01744381, 0.014799304, 0.013669492, 0.05338675, 0.02041112, 0.018635701, -0.010832544, 0.01955445, 0.016264336, -0.03560772, -0.026171923, 0.0025389746, 0.015792547, -0.011596099, 2.4346065e-05, 0.026445065, -0.0144765, -0.051300943, 0.007120303, 0.016078103, 0.012272745, 0.026271248, 0.07429449, 0.012539679, -0.010565611, -0.04412477, -0.03463931, 0.04933929, 0.0025979483, 0.0068471613, -0.025066942, -0.042560413, 0.0038798512, -0.013371519, 0.017642459, -0.029250974, 0.0028043564, -0.012210667, -0.0011344685, -0.018002508, -0.0028493626, -0.02773628, -0.0038426046, 0.004162304, -0.01220446, 0.01270108, 0.043007374, -0.02360191, -0.027587293, 0.043578487, -0.0033677109, -0.006859577, -0.0016497127, 0.0013199253, -0.03764387, 0.0043733683, -0.008274946, -0.0016248816, -0.006530565, 0.015444911, 0.019964162, 0.03389438, -0.035980187, -0.006636097, -0.0054566227, -0.013408765, -0.015109692, -0.016202258, 0.055224247, 0.0037308647, 0.0059253084, 0.014737226, -0.02704101, 0.024110947, 0.016090518, 0.010273846, -0.007505184, -0.0211933, 0.016785787, -0.0014114898, -0.013458427, 0.017853523, -0.029524116, -0.021031898, -0.062772885, -0.02088291, -0.0009071092, 0.0046247826, -0.0036160212, 0.0075920927, 0.01481172, -0.049686924, 0.016053272, -0.013557752, -0.023043212, -0.019020582, 0.012322407, 0.027562462, 0.005599401, -0.031361613, -0.0026522663, -0.008690867, 0.0145882405, 0.027115503, -0.004637198, 0.01849913, -0.004708587, -0.01586704, 0.016909944, 0.04362815, 0.01084496, 0.0014200255, -0.00812596, 0.008374271, 0.048048075, -0.04025113, -0.0068471613, -0.006096022, -0.0077286633, 0.011558852, -0.048395712, -0.02096982, -0.0037339686, 0.0031426793, 0.02452066, -0.009280603, 0.018052172, 0.0045968476, 0.029300636, -0.0056490633, 0.0073561976, -0.006148788, 0.014737226, -0.03106364, 0.02609743, -0.025141435, -0.0059191007, -0.005959451, 0.0067292135, -0.029822087, -0.0219134, -0.030467695, -0.017158253, 0.042312104, 0.0017986989, -0.0022627292, -0.00022056953, 0.0043609524, -0.028108746, 0.01885918, -0.00156358, -0.0020470095, -0.02157818, 0.0015721157, 0.029697932, 0.002891265, -0.0405491, 0.005081053, -0.012173421, 0.01597878, -0.016090518, -0.021031898, 0.028804015, -0.010466286, 0.007008563, -0.034068197, 0.009951042, -0.0023636052, 0.009497875, -0.031013979, 0.039580688, 0.019616527, 0.038314305, 0.0013393246, 0.031659584, 0.025575979, -0.00023220909, -0.013992295, 0.011701631, 0.009522706, 0.019641358, -0.009690316, -0.0075424304, -0.05810465, 0.018722609, 0.008995047, -0.021329869, -0.00027508143, -0.008163206, -0.02499245, 0.004019526, -0.005164858, -0.0028772976, 0.015345587, 0.018312898, 0.004503731, -0.036352653, -0.039357208, 0.036427148, -0.0118133705, -0.028133577, -0.025340084, 0.010664934, 0.019840006, 0.06252458, 0.002082704, 0.01650023, -0.0023092874, 0.023068042, -0.012458978, 0.013346688, -0.008243907, -0.018623285, 0.007424483, 0.0073561976, 0.01780386, -0.0006359076, -0.006179827, 0.006549189, -0.012936976, 0.029077156, 0.0023061836, -0.036129173, 0.03302529, 0.037817683, -0.013073546, -0.0066236816, 0.0027546943, 0.02728932, -0.021627842, -0.017642459, 0.0071078874, 0.0012493121, -0.0022611772, 0.021391947, -0.008008013, 0.014339929, -0.0014999504, -0.017766614, 0.010758051, -0.015569067, -0.012483809, -0.014079204, -0.0026196754, -0.030765668, 0.02052286, -0.0040443568, -0.006958901, 0.016909944, 0.005164858, 0.011428489, 0.016214674, 0.027860435, 0.05745904, 0.012552095, -0.009951042, 0.020796003, 0.015718052, -0.017481057, 0.009274396, 0.03126229, 0.03247901, -0.020249719, -0.0036377483, 0.0057918415, -0.00015199317, 0.0059035816, 0.045962267, -0.014265437, -0.005810465, 0.013793646, -0.025700133, -0.005884958, 0.0014192496, 0.0071078874, 0.0012050817, -0.015097276, 0.0026507142, -0.0064995266, 0.015581482, 0.018176327, 0.00017517527, -0.020088317, 0.014190943, 0.002661578, -0.013818477, -0.0032218283, -0.029946243, 0.0011430042, -0.035135932, -0.021900984, 0.0133094415, 0.00016217778, -0.007523807, 0.048271555, 0.016760956, -0.04434825, -0.017965263, -0.01292456, -0.001067735, 0.00776591, 0.020622185, -0.0011236048, -0.03747005, -0.01411645, -0.0035539437, -0.009398552, 0.007002355, -0.030939484, 0.033745393, -0.014550993, 0.029499283, 0.0024877605, 0.030269047, -0.0336709, -0.013259779, 0.02560081, -0.014898628], "content_hash": "f14a8f80b705fd4fecd313a1566dcc86"}, {"content": "is split into tokens 2. **Embedding**: How tokens become number vectors 3. **Attention**: How the model finds relationships between tokens 4. **MLP**: How stored knowledge is retrieved 5. **Output**: What the model predicts and how confident it is Click any stage to expand it and see detailed explanations and visualizations.\n\n### 5. Investigation Panel\nAt the bottom, two experiment tabs let you investigate *why* the model made its prediction:\n- **Ablation**: Remove specific attention heads and see what changes\n- **Token Attribution**: Measure which input tokens influenced the prediction most\n\n### 6. AI Assistant (Chatbot)\nThe floating robot icon in the bottom-right corner opens the AI chatbot. It can answer questions about transformers, explain what you're seeing in the dashboard, and guide you through experiments.\n\n## Typical Workflow\n\n1. Select a model (start with GPT-2 if you're new)\n2. Enter a prompt (e.g., \"The cat sat on the\")\n3. Click \"Analyze\" to run the model\n4. Explore the 5 pipeline stages to understand how the model processed your input\n5. Use the Investigation Panel to run ablation or attribution experiments\n6. Ask the chatbot if anything is unclear", "source_file": "dashboard_overview.md", "chunk_index": 1, "embedding": [-0.0012277357, 0.014412472, 0.009448724, -0.01637685, 0.0013958341, -0.044187922, -0.019488873, 0.05841029, -0.003749214, 0.069844514, 0.023825992, -0.0530593, -0.006393026, -0.046046685, -0.027247809, -0.014743389, -0.03548552, 0.021770084, -0.012349524, 0.054185826, 0.025304554, -0.01123004, 0.018700305, 0.053735215, 0.0047454843, -0.0017469929, -0.014560328, 0.012265035, -0.020840703, 0.013461967, 0.018404594, -0.022924772, -0.052214406, -0.03711898, -0.0016581031, -0.01068086, -0.021023761, 0.023051506, -0.004118855, 0.038639788, 0.016785214, 0.0090051545, -0.050017685, 0.052186243, -0.013659108, 0.0063120574, 0.009188215, -0.024769455, -0.035091236, 0.054016847, -0.036443066, 0.040695693, -0.019305812, 0.0004147458, -0.0074843466, 0.029317798, -0.008484137, -0.03250023, -0.02726189, -0.0099204555, 0.022051716, 0.005938896, 0.014011147, 0.027008424, 0.0040414063, -0.012983194, -0.034246344, 0.023572523, -0.037090816, -0.027994132, 0.026402917, -0.0027828673, -0.02219253, 0.022938853, 0.003367252, -0.003911152, -0.03359859, 0.050017685, -0.021333557, -0.037147142, 0.014616654, -0.045765053, -0.010195046, -0.041146304, 0.0007406018, -0.015813587, -0.035091236, -0.023924563, -0.06364863, -0.078068145, -0.018714387, 0.013659108, -0.059086207, 0.021206822, 0.06657759, 0.017165415, 7.937376e-05, -0.004914463, 0.0077800592, 0.033964712, 0.036696535, -0.023657013, -0.03821734, 0.012518503, 0.029937387, -0.021080088, 0.030669628, -0.00081012945, -0.04280793, -0.025304554, -0.097106405, -0.0057347133, -0.01657399, -0.0031296257, 0.0039146724, -0.00787159, 0.0042420686, 0.0043969657, 0.003478144, -0.05159482, -0.046553623, -0.006681698, 0.032218598, 0.008343321, 0.0009751477, -0.0007023176, 0.013180336, -0.02698026, 0.0069950125, 0.013546457, -0.04860953, -0.003303885, 0.040808346, -0.045849543, 0.0087235235, -0.023938645, 0.008737605, 0.024037214, -0.046891578, 0.022319265, 0.0067556263, -0.042216502, -0.014159004, 0.0004173861, -0.06663392, -0.024135785, -0.021389883, 0.058184985, -0.0715343, 0.011131469, 0.0152080795, 0.009293826, -0.04128712, -0.024107622, -0.022981098, -0.015067264, -0.019390302, 0.06421189, 0.0066887387, -0.062691085, 0.016010728, 0.022051716, -0.03207778, -0.017278068, -0.09045991, 0.0477928, -0.029402288, 0.011941159, -0.037963875, -0.053284604, -0.023234567, -0.029261472, -0.012349524, -0.034527972, 0.015883995, -0.031711664, -0.068380035, -0.040808346, -0.024290683, -0.04238548, -0.021784166, -0.01637685, -0.03495042, -0.024910271, -0.018672142, -0.017770924, -0.015982565, 0.022009471, -0.027233727, -0.0004418088, -0.00084753364, 0.034612462, 0.033908386, 0.010709023, 0.0228262, 0.010420351, 0.044018943, 0.0110469805, 0.040047944, 0.012441054, 0.05931151, 0.053481746, 0.039541006, -0.033204306, 0.0432867, 0.03069779, -0.0041610994, 0.01574318, -0.04455404, -0.012081974, 0.02752944, -0.066971876, 0.03537287, 0.008132098, -0.07198491, -0.0022160849, -0.08432035, 0.04424425, 0.014278697, 0.03531654, 0.013954821, 0.017855411, 0.014574409, -0.009293826, -0.02798005, 0.024558233, 0.03658388, -0.044976488, 0.0175597, -0.002099912, -0.002399145, 0.02579741, -0.032979004, -0.009561376, -0.011455345, -0.04354017, 6.4797154e-05, -0.017770924, 0.025276393, -0.03207778, 0.026825363, 0.005861447, 0.040498555, 0.0102020865, 0.020052135, 0.013250744, 0.00035071874, -0.036414903, 0.061958842, 0.029458614, 0.061170276, -0.019207241, -0.033204306, -0.007343531, 0.02453007, -0.03402104, -0.020671723, 0.02789556, -0.022558652, 0.0042667114, 0.007913834, 0.030500649, -0.05080625, -0.022248857, -0.021389883, -0.007188634, -0.030894933, -0.0069879717, -0.046046685, -0.071590625, 0.017052764, -0.0025293995, 0.00033751727, 0.010617493, -0.062071495, -0.015771342, 0.008660156, 0.01575726, -0.018306023, 0.01874255, -0.017278068, -0.033626754, 0.049172793, 0.028008213, -0.011920037, 0.007097104, 0.0037597753, -0.011307489, 0.011307489, -0.0059705796, 0.03410553, 0.000642911, 0.03421818, -0.02181233, 0.01829194, 0.0078082224, 0.038329992, 0.036274087, 0.007385776, 0.013067683, 0.02671271, -0.05514337, 0.022755794, -0.039597332, 0.011103306, 0.026754955, -0.02409354, -0.013546457, 0.023234567, -0.02136172, 0.00037426135, -0.021798247, 0.009209337, -0.013229622, 0.01249034, 0.016320523, 0.04455404, 0.00069087633, 0.036020618, -0.028726373, -0.03528838, -0.051200535, -0.021333557, -0.013785843, -0.007836386, -0.020502744, -0.01204677, 0.031176563, -0.017306231, -0.0190101, 0.039907128, -0.04762382, -0.029148819, 0.014447676, -0.013652068, -0.012469217, -0.014799715, 0.016236033, -0.025318636, 0.048637692, -0.072210215, -0.015222161, -0.06578903, -0.0093219895, -0.06139558, 0.06319802, 0.015729098, 0.010223209, 0.0013271866, -0.0031542685, -0.017066846, -0.007181593, 0.0031349063, -0.029036166, 0.006854197, 0.020756213, -0.016785214, -0.026923934, 0.018629897, -0.009188215, -0.03486593, 0.03204962, 0.06961921, 0.030754117, -0.024192112, -0.0019520556, -0.039794475, 0.011687691, -0.010610452, -0.02334722, 0.015320732, 0.009420561, 0.038780604, -0.0045765056, -0.043934453, -0.028346172, 0.009652906, 0.0035573528, -0.041512426, 0.03883693, 0.039428353, -0.013715435, -0.025684757, -0.03207778, 0.02017887, 0.029176982, 0.0051644105, -0.007128787, -0.042047523, 0.053284604, 0.011835547, -0.059255186, -0.026740873, -0.0006886761, 0.024670884, 0.04100549, 0.0006715142, 0.015306651, 0.051200535, 0.051566657, 0.012814215, -0.03540103, -0.049736053, -0.018263778, 0.011089224, 0.026684547, 0.037766732, -0.013243703, 0.030556975, 0.009786681, 0.051397678, -0.05429848, 0.022248857, -0.0036858472, -0.03883693, 0.01784133, 0.036499392, -0.0019045303, 0.044976488, 0.045229957, -0.014180127, -0.029458614, -0.022798037, -0.028360253, -0.062353127, 0.024009053, 0.051116046, -0.010019027, -0.0010332342, -0.02816311, 0.030106366, 0.04255446, 0.055171534, -0.007171032, 0.021685595, -0.03920305, 0.045793217, 0.033880223, 0.0045377812, 0.020629479, 0.027839234, -0.034387156, -0.023248648, -0.02798005, -0.054073174, 0.0005306986, -0.03188064, -0.072998784, -0.008096894, -0.03849897, 0.007252001, -0.01584175, -0.0028831984, 0.05421399, -0.030303506, -0.08015221, 0.044694856, 0.013828088, 0.080771804, 0.010927287, 0.01910867, 0.027543522, -0.007378735, 0.02779699, -0.006209966, -0.002796949, -0.02869821, -0.015517874, 0.000113422524, 0.010885042, -0.008195465, -0.036161434, -0.013264825, -0.025895981, 0.007033737, -0.030387996, 0.02752944, -0.01376472, 0.007822304, -0.041315284, 0.014715225, -0.0053826747, -0.015588282, 0.02906433, 0.00057558354, 0.07987058, -0.035457358, -0.022108043, 0.02959943, -0.008477096, -0.01050484, -0.016996438, 0.03503491, -0.017587863, 0.045765053, -0.049820542, -0.037090816, -0.022389673, -0.009723314, 0.014687062, 0.0076181213, -0.012117178, -0.0039674784, 0.086798705, -1.7051883e-06, -0.023558442, 0.03666837, 0.034753278, 0.0024501907, 0.022727631, -0.035907965, -0.0064211893, 0.0100260675, 0.020967435, -0.022333346, 0.021756003, 0.02943045, 0.025318636, 0.025107414, 0.004569465, 0.0060515483, 0.014144923, 0.018320104, -0.018629897, 0.016785214, -0.022093961, -0.038076524, -0.04058304, -0.06128293, -0.006512719, 0.009786681, -0.017517455, 0.011032899, -0.010089435, -0.050017685, 0.0013773522, 0.0005298185, 0.00025544822, -0.055565815, -0.0097796405, 0.014320942, 0.0037351325, 0.007533632, 0.004597628, -0.0097444365, 0.0085827075, -0.024684966, -0.012715644, -0.00072652026, 0.027557604, 0.045145467, -0.029374124, 0.018714387, -0.038273666, 0.0090403585, -0.0020471062, 0.017672352, -0.026022714, 0.019981727, -0.019136833, -0.002473073, 0.0036471228, 0.012800134, -0.039428353, -0.007216797, 0.02761393, 0.018207451, -0.02119274, -0.0015604123, -0.029712081, -0.01313105, -0.0071675116, -0.005541092, 0.018348267, 0.0077659776, -0.026585976, -0.0077448552, 0.016771132, -0.023121914, 0.0009082603, -0.014011147, -0.028050458, -0.040723857, 0.0075195506, -0.0034094965, 0.00029769287, -0.012349524, 0.009378316, -0.015827669, 0.014313901, 0.040216923, -0.02172784, -0.0030556975, 0.0020981517, -0.025698839, -0.003970999, 0.026642302, 0.00055446126, -0.024853945, 0.008209547, 0.020418255, -0.006871799, -0.020474581, 0.013025438, -0.03940019, 0.004235028, -0.025895981, -0.0019397342, 0.014067474, -0.017982146, 0.011631365, -0.05303114, -0.0075758765, -0.02072805, -0.024558233, 0.0114131, -0.020404173, 0.017137254, 0.032979004, -0.0027863877, -0.002481874, -0.006963329, 0.01032178, 0.0046328316, 0.0033074054, 0.017728679, 0.0019784586, -0.010173923, -0.009512091, 0.037794895, -0.011427182, 0.038527135, 0.032387577, 0.0037140101, -0.008110976, 0.019671932, 0.0036788064, 0.008322199, -0.007815263, -0.012610033, 0.007667407, -0.018348267, -0.007660366, -0.02381191, 0.004555383, 0.0059776204, -0.006674657, 0.017700516, -0.034556136, -0.023924563, -0.042667113, -0.038893256, 0.040921, -0.0063895057, 0.0076040397, -0.012532584, -0.0016325803, 0.00023982649, -0.0073364903, 0.02334722, -0.02752944, 0.0037104897, 0.016010728, -0.02081254, -0.019333975, 0.040808346, 0.013180336, 0.013328192, 0.017587863, 0.0036083986, -0.06494413, 0.015982565, -0.0154897105, 0.039541006, 0.0013711915, -0.02498068, -0.0038442647, -0.05007401, 0.017531537, 0.061170276, -0.022220694, -0.0077800592, -0.057734378, 0.005586857, 0.03675286, -0.0098148445, 0.01502502, -0.01882704, 0.02761393, -0.008885462, -0.007142869, -0.032556556, -0.013074724, -0.009519131, -0.024417417, 0.009758518, 0.0015586521, 0.048778508, 0.0003106743, -0.07913834, -0.014743389, -0.017095009, -0.027332298, -0.034190018, -0.018587653, -0.016982356, -0.017137254, 0.044328734, 0.025755165, -0.03241574, -0.042441808, -0.0008563346, 0.012032689, 0.01865806, -0.03159901, 0.030613301, -0.0008193705, -0.0027529441, -0.015475629, -0.020333765, -0.009244541, 0.010948409, -0.016278278, 0.017616026, 0.015644608, -0.06533842, -0.040329576, 0.0077448552, -0.027628012, 0.018066635, 0.010687901, 0.018798877, -0.0012629395, -0.067591466, 0.027402706, -0.0018394032, -0.0089699505, -0.006924605, -0.037344284, -0.027599849, -0.0024132265, 0.023431707, 0.0042596706, 0.02037601, 0.022952935, 0.003539751, -0.02182641, 0.039541006, 0.0055727754, -0.04390629, -0.0022706508, 0.013102887, -0.026191693, -0.015222161, -0.011962282, -0.0008092494, 0.014475839, -0.041146304, 0.00036788062, 0.035654496, -0.012384728, 0.015053183, -0.0023287372, 0.0104133105, 0.0031771509, -0.027937805, 0.03731612, 0.012243913, 0.004928544, 0.017616026, -0.0074209794, -0.0092515815, 0.00017645949, -0.027656175, 0.06139558, -0.018587653, 0.03359859, -0.010279535, 0.00579456, -0.022220694, -0.019573363, 0.009026277, -0.007477306, -0.036076944, -0.0055270107, 0.0010261934, -0.015334814, -0.007100624, -0.021023761, -0.026135366, -0.037794895, 0.00019989208, 0.011905955, -0.040216923, -0.008815054, -0.001973178, 0.003946356, -0.10302066, -0.015095428, 0.003251079, -0.008406688, 0.007315368, -0.03233125, 0.049736053, 0.011152592, -0.023966808, -0.030162692, 0.015109509, -0.032359414, 0.0032352374, -0.0023111352, 0.010349943, -0.023051506, 0.01764419, -0.016771132, 0.0042667114, -0.017785005, -0.018418673, -0.0056185406, 0.009469846, 0.01747521, -0.04438506, 0.012187586, -0.017489292, 0.034556136, 0.018841121, 0.0015190478, -0.038414482, 0.013335233, -0.012265035, 0.021319475, 0.007927916, 0.031373706, -0.013426763, 0.01357462, -0.0037844179, -0.002934244, 0.006681698, -0.02017887, 0.043089557, -0.016447257, 0.05097523, -0.017630108, -0.00452722, -0.031655338, 0.0050940025, 0.024487825, -0.028839026, 0.013687272, -0.01023025, -0.001484724, 0.01910867, 0.021164577, -0.014658899, -0.012201668, 0.0052559404, 0.006273333, 0.00452722, 0.025276393, 0.020756213, 0.010173923, 0.0044884956, 0.026205774, 0.013145132, 0.010054231, -0.039315704, -0.024248438, -0.0058825696, -0.008357403, 0.0058825696, -0.019263567, -0.0075758765, 0.005305226, 0.0002462072, 0.032472067, 0.0025100373, -0.010638615, 0.027726583, -0.0023657014, -0.045962196, 0.015405222, 0.007632203, 0.010314739, -0.009758518, -0.03151452, 0.015433385, 0.022769876, 0.029374124, -0.00026666946, 0.00063322997, -0.021488454, -0.019446628, -0.0032035538, 0.0001113873, 0.04500465, 0.052552365, -0.016503584, 0.04472302, 0.020361928, -0.0327537, 0.0016220191, -0.021460291, 0.02988106, -0.016883785, -0.011800343, -0.017404802, -0.0077730184, 0.020544989, 0.0022512886, -0.0040766103, 0.0113638155, -0.019122751, 0.02996555, 0.03393655, 0.008624952, 0.011934118, -0.033880223, 0.004928544, 0.014715225, -0.034077365, 0.008075772, -0.023065588, 0.0038055403, -0.0070583797, 0.021418046, 0.023488034, -0.0014812036, -0.012257994, 0.03033167, -0.0051292065, -0.00072652026, -0.027318217, 0.009498009, 0.016700724, 0.018249696, -0.028275764, 0.0015604123, -0.004203344, 0.018306023, -0.011004736, 0.008040568, -0.05522786, 0.0018358828, -0.007857508, -0.01612338, -0.004030845, 0.016306441, 0.021206822, 0.013109928, -0.0038970704, 0.025853736, -0.008286995, 0.016630316, -0.00095050497, -0.0390904, -0.005657265, 0.029852897, 0.012511462, 0.037259795, 0.04599036, -0.00606563, 0.017672352, -0.004509618, -0.0044216085, -0.00033267675, -0.0024132265, 0.008491178, -0.02988106, 0.033373285, 0.05190461, -0.0008998994, 0.045934033, 0.02943045, 0.021178659, 0.031908803, 0.003946356, 0.034753278, -0.005259461, 0.058072332, -0.0087235235, -0.0030169731, 0.013145132, -0.021784166, -0.00479829, 0.026191693, -0.0020066216, 0.010223209, 0.008385566, -0.009758518, -0.008800972, 0.04255446, -0.030725954, -0.0018886887, 0.018404594, -0.004238548, -0.026867608, 0.004118855, -0.014996856, 0.021629268, 0.030585138, -0.023417626, 0.012786052, 0.011004736, -0.0041223755, 0.0070583797, -0.015010938, -0.011300448, -0.027656175, -0.0019115711, -0.002143917, 0.035513684, -0.012060852, 0.015517874, 0.011124428, 0.012722685, 0.008251792, -0.0060163443, -0.04889116, -0.039259378, -0.003721051, -0.008455974, -0.019348057, 0.019854993, -0.0035819956, 0.016348686, -0.009293826, 0.0040449267, 0.008244751, -0.012821256, -0.0006723943, -0.0012629395, -0.008364444, 0.005981141, 0.004998952, 0.00059538573, 0.039822638, -0.01657399, 0.013982984, 0.019446628, -0.006100834, 0.008026486, -0.011582079, -0.0013711915, 0.033091653, 0.008702401, -0.0012338962, -0.026149448, 0.0064880764, 0.017179497, 0.026093122, -0.01603889, 0.020868866, -0.018629897, 0.052439712, 0.024952516, -0.02136172, -0.011406059, 0.011652487, 0.031035747, 0.013905535, -0.030528812, 0.0046046684, -0.015123591, -0.023741502, -0.016067054, 0.0061606807, 0.0017487531, -0.025304554, 0.036893673, -0.01874255, 0.018869285, -0.004551863, 0.0020400654, -0.010716064, -0.020390091, -0.0007014375, -0.02262906, -0.020052135, -0.03148636, 0.0060726707, -0.0098148445, -0.005143288, -0.0103429025, -0.031430032, -0.030838607, 0.00096546666, 0.0034535015, -0.0032352374, 0.020939272, 0.000371181, -0.011934118, -0.0067415447, 0.017038682, 0.010040149, -0.012701563, 0.000851494, -0.031739827, 0.018967856, -0.011448304, -0.0053122668, -0.04018876, 0.018728469, -0.00534043, 0.0058332845, -0.036443066, 0.017080927, 0.003794979, -0.049820542, 0.008976991, 0.00950505, 0.025473533, 0.017700516, 0.0022231257, 0.0048475754, -0.037372448, 0.015433385, -0.008547504, -0.018784795, 0.012187586, 0.008674238, 0.025839655, -0.011934118, -0.02634659, 0.02781107, -0.006393026, 0.014701144, 0.008441892, 0.014151963, -0.009124848, 0.00090562005, 0.014313901, -0.03917489, -0.015081346, -0.049933195, -0.020953353, 0.00035093876, 0.0015102468, 0.020939272, 0.018235615, -0.021080088, -0.005967059, 0.0133986, -0.0090051545, 0.022896608, -0.025473533, 0.02761393, 0.009110766, 0.070464104, -0.029543104, -0.034443483, -0.003096182, 0.01765827, 0.0072379196, -0.007927916, 0.0031841916, -0.03838632, 0.0039252336, 0.0009373035, 0.016644398, -0.0092163775, 0.009089644, -0.0069210846, -0.011349734, 0.00074104185, 0.009765559, 0.046553623, 0.0017601944, 0.011145551, 0.046046685, -0.014785633, -0.004312476, 0.0044673732, 0.013440845, 0.0071217464, -0.020657642, 0.016052973, 0.017221741, -0.0072942455, -0.056157243, 0.0052911444, 0.029374124, -0.008639034, 0.002951846, -0.02624802, 0.023994971, -0.023417626, 0.011166673, 0.021249067, -0.005868488, 0.012610033, -0.028219437, -0.021347638, 0.015644608, -0.01620787, -0.039372027, -0.015179916, 0.001955576, 0.0061888434, 0.024318846, 0.003133146, -0.0018112401, 0.020432336, -0.03188064, -0.015855832, 0.03207778, 0.0148560405, 0.0063366997, -0.03849897, 0.043709148, 0.001964377, 0.03557001, 0.009209337, -0.040780183, -0.01765827, 0.014644817, 0.005910733, 0.02444558, -0.016278278, 0.0003577595, 0.004682117, -0.0083010765, 0.032810025, 0.028923515, 0.017376639, -0.014912367, 0.024248438, 0.0076392437, -0.022474162, -0.004463853, -0.000498135, -0.002606848, 0.011603202, -0.0075547546, 0.044131596, -0.0041681402, -0.025769247, 0.022037635, 0.0012787812, 0.021220904, 0.018756632, 0.044413224, 0.0064810356, 0.000552261, -0.023417626, -0.0053545116, 0.024065377, -0.01765827, -0.009934537, -0.0017056284, -0.044610366, -0.009181174, -0.012567788, -0.00425263, -0.03773857, 0.011905955, 0.020573152, 0.022417836, 0.014335023, 0.014489921, 0.00033509702, 0.0008382926, 0.00031309458, 0.012194627, 0.014701144, 0.054101337, -0.02634659, -0.036724698, 0.019179078, -0.01539114, 0.004066049, -0.01357462, 0.0033109258, 0.010842797, 0.0151376715, -0.015827669, -0.022474162, 0.0046785967, -0.008815054, 0.031852476, 0.018066635, -0.044441387, -0.0009795482, -0.0013254264, -0.014602573, -0.021023761, -0.03866795, 0.057734378, -0.008413729, 0.032359414, 0.031120237, -0.04111814, 0.043878127, 0.00076612463, 0.016334604, 0.008463015, -0.01285646, 0.010892083, -0.022333346, -0.016672561, 0.04289242, 0.01249034, 0.0112934075, -0.04970789, -0.010800553, -0.023938645, -0.013363396, -0.005748795, 0.030219018, -0.003567914, -0.039118562, -0.0052488996, 0.0006979171, 0.0089347465, -0.012025648, 0.039541006, 0.034358993, 0.0043265577, -0.009026277, -0.0036330412, -0.0090403585, 0.022474162, 0.016827459, -0.01376472, 0.029909223, 0.019854993, -0.0335141, -0.002110473, 0.006502158, 0.0030275343, -0.028036376, -0.013870332, -0.0096740285, 0.040949162, -0.033542264, -0.011898914, -0.021150496, -0.008392607, -0.02634659, -0.039372027, -0.029993713, 0.0032810024, 0.025149658, 0.030387996, -0.024079459, 0.029345961, 0.032922678, 0.003384854, -0.020023972, -0.005125686, -0.017123172, -0.019094588, -0.025135577, 0.06708453, -0.012891664, -0.018320104, 0.0061888434, 0.033570427, -0.023319056, -0.014644817, -0.025262311, -0.01629236, 0.030641465, -0.012469217, -0.013771761, -0.019066425, 0.0038020199, -0.025205985, 0.0022301662, -0.008463015, 0.015771342, -0.022220694, -0.00056106196, 0.011244122, -0.014151963, -0.04044223, 0.042329155, -0.00082773145, 0.019446628, -0.018756632, -0.0036189598, 0.034471646, -0.0100260675, -0.0011089225, -0.00031551486, -0.01430686, 0.01439135, 0.023445789, -0.027754746, 0.01375768, 0.023150077, 0.029683918, -0.014159004, 0.017235823, 0.010511881, 0.0038513055, -0.043878127, 0.012624115, 0.028543312, 0.03033167, -0.021798247, -0.018094799, -0.006871799, -0.01739072, 0.001366791, -0.0005034156, -0.0034728635, -0.016010728, -0.032838188, -0.001828842, 0.0029553664, -0.017799085, 0.011835547, 0.03086677, -0.00019208122, -0.013349314, -0.020756213, 0.016052973, -0.01882704, -0.019700095, 0.004365282, -0.014898285, 0.004544822, 0.03478144, 0.0036858472, -0.006935166, 0.0018570051, -0.028543312, -0.018235615, -0.001167889, -0.0051045637, -0.0058544064, 0.00298705, 0.0056643058, 0.018362349, 0.00068647583, -0.005298185, -0.0060339463, 0.008012405, 0.03213411, -0.007533632, -0.014813797, 0.014644817, 0.0016581031, -0.003467583, -0.022009471, -0.011391978, 0.049764216, 0.008068731, -0.010603411, 0.03703449, -0.0016096978, 0.037513264, 0.036161434, -0.008040568, 0.034612462, -0.0070407777, -0.005414358, 0.0010622774, -0.025952306, -0.006530321, -0.01774276, 0.008153221, -0.020080298, 0.007273123, -0.012715644, 0.00040066426, 0.010814634, 0.02781107, 0.0074561834, 0.0029448052, 0.0079842415, 0.046412807, -0.00029021205, 0.0048581366, 0.0068577174, 0.027106995, -0.026698628, 0.006484556, 0.04770831, 0.02472721, -0.0013747119, 0.004400486, -0.0054706843, -0.001503206, -0.003731612, 0.01729215, -0.013891454, -0.032387577, -0.006417669, -0.008343321, -0.022868445, -0.013384518, 0.038977746, -0.007068941, -0.028923515, -0.0075688357, -0.013116969, 0.01584175, 0.043089557, -0.0027617451, -0.018249696, -0.015729098, 0.014060433, -0.0008668958, 0.014067474, -0.004629311, -0.016855622, -0.049848706, -0.010420351, -0.004509618, -0.0040695695, 0.008125057, 0.052355222, -0.025487615, -0.039597332, -0.01639093, -0.016405012, -0.009723314, 0.0041153347, 0.045934033, 0.004354721, -0.023657013, -0.010195046, -0.032105945, -0.012117178, -0.003423578, -0.01276493, 0.042244665, -0.03385206, 0.020361928, 0.008019445, 0.02996555, -0.01692603, -0.02081254, 0.013539416, 0.016897867], "content_hash": "64dd8abf7a9430ef70ceb42f45496444"}, {"content": "# Embeddings Explained\n\n## What Are Embeddings?\n\nAfter tokenization breaks your text into token IDs, the model needs to convert those IDs into something it can actually compute with. This is where **embeddings** come in. An embedding is a list of numbers (a **vector**) that represents a token's meaning.\n\n## The Lookup Table\n\nThink of embeddings as a giant dictionary. Each token ID maps to a specific vector of numbers. For GPT-2, each token maps to a vector of **768 numbers**. For larger models, this vector might have 2048, 4096, or more numbers.\n\nThis dictionary (called an **embedding table**) was learned during training. The model figured out which numbers best represent each token by seeing how tokens are used across billions of text examples. Once training is done, the table is fixed -- the same token always maps to the same vector.\n\n## Why Vectors?\n\nWhy use lists of numbers instead of just the token ID? Because vectors let the model capture **meaning** and **relationships**:\n\n- Words with similar meanings (like \"happy\" and \"joyful\") end up with similar vectors\n- Related concepts are grouped nearby in the vector space\n- Directions in the space can capture relationships (e.g., \"king\" - \"man\" + \"woman\" \u2248 \"queen\")\n\nThis is what allows the model to generalize -- even if it has never seen a specific sentence before, it can work with the underlying meanings of the tokens.\n\n## Positional Information\n\nThere's one more detail: the model also needs to know the **order** of tokens (since \"dog bites man\" is different from \"man bites dog\"). This is handled by adding **positional encodings** to the embeddings -- extra numbers that tell the model where each token sits in the sequence.\n\n## What You See in the Dashboard\n\nIn **Stage 2 (Embedding)** of the pipeline, you can see:\n\n- The dimension of the embedding vectors (e.g., \"768-dimensional\" for GPT-2)\n- A visual showing the flow: Token ID \u2192 Lookup Table \u2192 Vector\n- An explanation of how the lookup table was created during training", "source_file": "embeddings_explained.md", "chunk_index": 0, "embedding": [-0.03326277, 0.00024296207, 0.021660032, 0.0074122003, 0.009177576, -0.013540491, 0.0023300583, 0.03411871, -0.019603398, 0.0581088, 0.03644877, 0.0015246427, -0.0016360931, -0.08502335, 0.0018902003, -0.043415166, -0.014967058, 0.027366295, -0.010259389, 0.06158011, 0.02056633, -0.025012461, -0.01836704, 0.062340945, -0.013290842, 0.017998511, 0.015466356, 0.049406745, 0.0172139, 0.015906213, 0.0022602158, -0.021505486, -0.029529922, -0.036876738, 0.015739782, -0.023776105, -0.013445387, 0.03166977, -0.029553698, 0.04916898, 0.02563064, 0.032573264, -0.06067662, 0.053353578, -0.029434817, 0.03307256, -0.019853046, -0.03533129, -0.031336904, 0.04436621, -0.00096961926, 0.029101951, -0.03751869, 0.004597704, 0.01129365, 0.009290513, -0.055683635, -0.00986114, -0.028364893, -0.040942453, 0.01903277, 0.006116403, 0.04774242, 0.012423014, 0.013433499, -0.018283824, -0.073087744, 0.06433814, -0.03673408, -0.0055636084, 0.059345156, 0.03326277, -0.033357874, 0.02819846, 0.04916898, -0.024465611, -0.03678163, 0.060153544, -0.05582629, -0.07361082, 0.0065443725, -0.049216535, -0.016678937, -0.0065681487, -0.016441176, -0.035830587, 0.0072398237, -0.024679596, -0.004734417, -0.03797044, -0.052782953, -0.024299178, -0.054780144, -0.007358704, 0.014919505, 0.010075124, 0.0044342433, -0.027294967, -0.023288693, 0.0080244355, 0.008702054, -0.007097167, -0.030647399, -0.022254433, 0.020340458, -0.05730041, -0.014206222, 0.017487325, -0.032264173, -0.055493426, -0.09985964, 0.016393624, -0.0066751414, 0.019627174, 0.033215217, 0.013813917, -0.03742359, 0.046601165, -0.0032365222, -0.0092191845, -0.017297115, -0.043177404, 0.042321466, 0.0127321035, -0.02148171, 0.012000988, -0.025178894, 0.023027157, -0.013885245, -0.020245353, -0.032454383, -0.010895399, 0.032501936, -0.026581684, -0.042036153, -0.026962101, -0.017558653, 0.041037556, -0.0016762153, -0.004603648, -0.017439773, -0.05340113, 0.007852059, 0.00017637039, -0.042868316, 0.0101761725, -0.016179638, 0.061485004, -0.022967717, -0.0077450657, 0.00592025, -0.0049424577, -0.0018604802, -0.031265575, -0.037185825, -0.031218024, -0.012244694, 0.03861239, 0.032858577, -0.02331247, -0.0046422845, 0.018533474, 0.049026325, -0.010853792, -0.069711536, 0.035426397, -0.010669527, -0.036829185, -0.008083875, -0.08064855, 0.028697757, -0.051736802, -0.026391476, 0.002514323, 0.036329888, -0.01996004, -0.048931222, -0.050928414, -0.0048532975, -0.059107397, -0.046648715, -0.0014926936, -0.028935518, 0.022718068, -0.03540262, 0.032430608, -0.009290513, 0.037138276, -0.012708328, 0.008511845, 0.006163955, 0.014075453, 0.045982987, 0.00547742, -0.016453065, 0.010978616, 0.02204045, -0.015299923, 0.06552694, -0.022563523, 0.03678163, 0.01281532, 0.03906414, 0.0013396349, -0.0035040034, -0.0065324847, -0.029434817, -0.009361841, -0.046220746, -0.0030329393, -0.008006603, -0.053496234, 0.007893667, -0.008874431, -0.023383798, 0.04260678, -0.05083331, 0.031218024, -0.01232791, 0.013492939, -0.0084107965, 0.027199863, 0.027080983, -0.0334292, -0.047219343, 0.0021487654, 0.00097184826, -0.063482195, 0.036211006, 0.0023419464, 0.016524393, -0.017653758, 0.012197142, -0.027699161, 0.013207626, -0.009801699, -0.009076527, -0.030599846, 0.00011637288, -0.0038012047, -0.02727119, 0.008505901, -0.0045085438, 0.027508952, 0.016191527, 0.007566745, -0.0117751155, -0.02983901, 0.06395772, 0.05016758, 0.02972013, 0.0055754962, -0.023978202, -0.009629322, -0.015656564, -0.019484518, -0.012898536, 0.028269788, 0.007893667, -0.021778911, 0.042868316, 0.012351686, -0.044746626, 0.0013440929, 0.0058132573, -0.0062828357, -0.023942538, -0.02886419, -0.03571171, -0.07147097, 0.021018077, -0.016334184, -0.008184924, 0.021065628, -0.06115214, -0.023728551, -0.04389069, -0.0051386105, -0.008333525, 0.015918102, -0.00012863244, -0.031146696, 0.05639692, 0.02582085, -0.017356556, -0.008143316, -0.004764137, 0.031764876, -0.013350283, -0.029577473, 0.03259704, -2.7003525e-05, 0.03723338, 0.012280358, -0.014289439, 0.044223554, 0.033643186, 0.019698502, 0.011448194, 0.00906464, 0.010705191, -0.07185139, 0.0021903736, 0.010645751, -0.020899195, 0.019139763, -0.0046779485, 0.016310407, -0.0045293476, -0.03568793, -0.014895729, -0.014325103, 0.007994715, -0.003459423, -0.0035961356, 0.03571171, 0.042273913, 0.026581684, -0.0063006678, -0.0495494, 0.012458678, -0.020174025, 0.011240154, -0.031027816, 0.015490131, -0.01427755, -0.04926409, 0.0147530725, 0.037185825, -0.015347475, 0.052830502, -0.0017445716, -0.023681, 0.02136283, -0.04700536, -0.029125728, -0.0034713112, 0.03925435, -0.0032721863, 0.016607609, -0.034190036, -0.016429288, -0.012957977, -0.03338165, -0.075655565, 0.031717323, -0.007911499, 0.02605861, 0.019674726, -0.0013106577, 0.004835465, -0.015644677, -0.0037269043, -0.04160818, 0.035307515, -0.0024712288, 0.01818872, -0.033928502, 0.008488069, -0.035854366, -0.02257541, -0.004648228, 0.017285228, 0.011156937, -0.02960125, 0.0042321463, -0.01470552, 0.028531324, -0.016453065, 0.01544258, 0.006027242, 0.0087971585, 0.009599602, 0.00224387, -0.018450256, 0.006116403, 0.036401216, 0.02765161, -0.033595636, 0.05577874, 0.024394283, -0.008987367, -0.031360682, -0.033595636, 0.0094034495, 0.049644504, 0.02245653, -0.01324329, -0.039325677, 0.0064849323, -0.03399983, -0.015311811, -0.0007742094, 0.0044699074, 0.026034834, 0.048931222, 0.014634192, 0.018331377, 0.026272595, -0.0022379258, -0.022147441, -0.035735484, -0.018783122, -0.03759002, 0.02093486, -0.025892178, 0.05121373, 0.00026023688, 0.034451574, 0.05316337, 0.010437709, 0.05344868, 0.017534876, -0.049692057, -0.0035485835, 0.019567734, 0.020292904, 0.019377524, 0.014301327, 0.04814661, -0.0023805825, -0.049596954, -0.07080524, 0.004832493, -0.015014609, -0.004324279, 0.050120026, -0.018176831, -0.014229998, -0.017927183, -0.011216377, 0.035473946, 0.05858432, -0.0047849407, -0.012084205, -0.01629852, 0.06543184, 0.03216907, 0.0049186815, 0.023467015, 0.007120943, 0.008440517, -0.015846774, -0.022777507, -0.062150735, -0.02142227, -0.03606835, -0.050215133, 0.017546766, -0.027722938, 0.029339712, 0.03778023, -0.004770081, -0.0026317176, -0.017285228, -0.03509353, 0.006360108, 0.00036109958, 0.06619267, -0.024560716, -0.015668452, 0.02691455, -0.032834798, -0.009801699, 0.002517295, -0.005944026, -0.038588617, -0.045816552, 0.022325762, -0.024941133, -0.0057865093, -0.0003754024, 0.013207626, -0.05045289, -0.011127217, -0.0053615114, 0.03338165, -0.023181701, -0.025487984, 0.018260049, 0.0018515642, -0.013267066, -0.045127045, -0.004327251, -0.0044342433, 0.060201097, -0.006651365, -0.018414592, 0.05563608, -0.02221877, -0.04253545, -0.042559225, -0.006342276, -0.01083596, 0.012648887, 0.008250308, -0.051689252, -0.056492023, 0.024132745, 0.015716005, 0.011715676, 0.0013322048, -0.0045025996, 0.06633533, 0.047694866, 0.028293563, 0.007762898, 0.025868401, -0.007548913, 0.042012375, -0.02734252, -0.0030611735, -0.002252786, 0.026700566, -0.004330223, 0.01879501, 0.019567734, 0.016571945, 0.009415337, -0.019436965, 0.006698917, -0.008624782, 0.024513163, -0.024489388, 0.0016672993, -0.010057292, -0.051308833, -0.036543872, -0.026819445, -0.008095764, 0.0007764384, -0.024073305, 0.011436306, 0.012024765, -0.059582915, 0.033191442, -0.013029305, -0.010437709, 0.005444728, 0.010978616, -0.01281532, 0.04852703, 0.021386605, 0.010948896, -0.00729332, 0.025654417, 0.0026658957, 0.005744901, -0.004817633, 0.009920579, -0.0017059355, -0.019211093, 0.034522902, -0.03471311, -0.03644877, 0.005310987, 0.011364978, 0.017629981, 0.019329973, -0.004505572, -0.0129342005, 0.010574423, 0.047266897, 0.00596483, -0.036115903, 0.06652554, 0.028531324, -0.020637657, 0.01824816, -0.045792777, -0.032668367, -0.016583832, 0.027675385, 0.044461314, -0.0042826706, -0.008173035, 0.00041756782, 0.0012824236, -0.034689337, 0.016464952, -0.022563523, -0.011281761, -0.0053228755, -0.010538758, -0.02788937, 0.014134894, 0.014206222, -0.008672334, -0.009694707, -0.005263435, 0.04619697, -0.012660775, -0.024061417, 0.028769085, 0.005741929, 0.0072636, 0.016191527, -0.027485177, -0.051118623, -0.016322296, 0.0060034664, -0.018509697, -0.034760665, 0.019817382, -0.020102696, -0.0055606365, -0.013659372, 0.02995789, -0.0064789886, 0.004814661, 0.0036882681, -0.05316337, -0.019211093, 0.015109714, -0.03411871, -0.0024935189, 0.0015677369, -0.0141824465, 0.00541798, 0.0060064383, 0.018973332, -0.007875835, 0.011721619, -0.04591166, -0.005319903, 0.047195565, -0.010711135, -0.02282506, -0.0095698815, 0.022123665, -0.005914306, -0.0027000739, 0.041156437, 0.005441756, 0.0037120443, 0.008202756, -0.015918102, 0.017463548, 0.029672578, -0.037756454, 0.00014432838, -0.027841818, -0.00035627006, -0.0048711295, -0.019555846, 0.0035040034, 0.0006817055, 0.03143201, -0.019318085, -0.024917357, -0.010289109, -0.052450087, 0.04167951, -0.023264918, 0.020340458, -0.034736887, -0.021885904, -0.01708313, -0.007715346, -0.010283165, -0.01665516, -0.02264674, -0.009641211, -0.027366295, -0.014907617, 0.039634764, 0.008826879, 0.013623708, 0.002707504, -0.0018738542, -0.07284998, 0.036353663, 0.014943281, 0.014432095, -0.0074775848, -0.036543872, -0.018557249, -0.006669197, -0.002985387, 0.02788937, -0.020281017, 0.008220588, -0.015727893, 0.006158011, 0.0064552124, -0.03478444, 0.029648801, -0.01525237, 0.01684537, -0.007863946, -0.017190123, -0.008838766, 0.008434573, -0.03642499, -0.020174025, 0.027128534, 0.0042380905, 0.022111777, 0.0023389743, -0.033476755, -0.0067167496, -0.008939815, -0.041275315, -0.053971756, -0.034998424, -0.019365637, 0.0054060915, 0.028364893, -0.00869611, -0.01330273, -0.0388026, 0.006651365, 0.008791215, 0.015965654, -0.014194334, 0.0028427304, -0.036353663, -0.009397505, 0.0035337235, 0.006140179, -0.037304707, 0.002995789, -0.009272681, -0.002749112, 0.011519522, -0.03509353, -0.053686444, -0.020768426, -0.014491536, 0.022860724, 0.016179638, 0.02496491, 0.012482455, -0.045816552, 0.017415997, 0.040562034, -0.018890115, 0.00958177, -0.029125728, 0.0015573348, -0.03352431, 0.027532728, 0.004781969, 0.0139803495, 0.002955667, 0.014764961, -0.0071803834, -0.003168166, 0.012672664, -0.039111692, 0.0035515556, 0.00789961, -0.0035069752, -0.037613798, 0.0040092454, 0.006811854, 0.02331247, -0.01189994, -0.014479647, 0.022313874, 0.012375462, 0.02245653, 0.009058695, 0.025345327, 0.027508952, -0.0451746, 0.020518778, 0.011650291, 0.024370506, 0.015680341, 0.011656235, -0.01324329, 0.0029304049, 0.0047671087, 0.052022114, -0.024334842, 0.04591166, 0.01806984, -0.01238735, 0.0066929734, -0.026534133, 0.025939729, 0.017689422, -0.04343894, -0.0014756045, 0.0046898364, -0.010289109, -0.015882438, -0.012292245, -0.05582629, -0.017249564, -0.028032027, 0.015822997, 0.0021784855, -0.019757943, -0.016441176, -0.0009681332, -0.08473804, -0.025464207, 0.044865508, 0.0097898105, 0.025773296, -0.00596483, 0.037304707, -0.006853462, -0.013385947, -0.033785842, -0.0034921153, -0.03808932, 0.026343923, -0.024608267, 0.01795096, -0.01187022, 0.013457275, 0.011632459, 0.008339468, -0.0014614874, -0.03568793, 0.013599931, 0.04051448, -0.010562534, -0.043010972, -0.00077123736, 0.011156937, 0.011067777, 0.03295368, 0.007982827, -0.04852703, 0.013516715, -0.01903277, 0.028769085, -0.0030210512, -0.002769916, 0.009023031, -0.022527859, 0.019246757, 0.001606373, 0.046220746, -0.017653758, -0.0067286375, -0.018283824, 0.058679424, -0.010711135, -0.0074775848, -0.00038301817, -0.0064611565, 0.020399896, 0.010669527, 0.029315937, -0.02453694, -0.015585236, 0.044817958, 0.0033762068, -0.00048666712, -0.021018077, 0.03478444, -0.002549987, -0.00789961, -0.008678278, 0.019389413, -0.00088417385, 0.020185912, 0.010211837, 0.026866999, 0.028935518, -0.029625027, -0.0005691405, -0.0023850405, -0.02031668, 0.0123992385, -0.023645336, -0.03825575, 0.010711135, 0.011864276, 0.013742588, -0.017166346, -0.01916354, 0.03416626, -0.006431436, -0.040966228, 0.042083703, 0.0037239322, 0.010752743, -0.025583088, -0.044461314, -0.008862543, 0.0040033017, 0.014562864, 0.0030849495, -0.028341116, 0.0046601165, 0.01101428, -0.0038130926, 0.006859406, 0.015609012, 0.047932625, -0.00918352, 0.06405283, 0.018105503, -0.022694292, -0.026866999, -0.06295913, 0.01336217, 0.016583832, -0.02551176, -0.007317096, -0.0072160475, -0.01885445, 0.0067048613, -0.024322955, -0.007079335, -0.06847518, 0.029197056, 0.01941319, 0.03350053, 0.023764215, -0.058631875, -0.0051148343, 0.0014978945, 0.00026822416, 0.028792862, -0.03069495, 0.020150248, 0.008143316, 0.039206795, -0.0030151072, 0.0028620486, -0.02117262, 0.05563608, 0.027057206, -0.017059354, -0.032644592, 0.01782019, -0.012613223, 0.009825475, -0.047433328, 0.0066929734, -0.020352345, -0.0036258558, -0.020292904, -0.004585816, -0.0039290013, 0.033928502, -0.037114497, -0.023217365, -0.009385617, -0.004160818, 0.040062737, 0.024085194, -0.036282334, 0.049121432, 0.018557249, -0.0010461486, 0.010889456, -0.045245927, -0.017154459, 0.029458594, 0.010324773, 0.024251627, 0.011311482, 0.0060866824, 0.043581598, -0.010663583, -0.015490131, 0.016821593, -0.028293563, -0.0026272596, -0.01616775, 0.028246012, 0.001821844, 0.03625856, 0.033286545, 0.035545275, -0.018283824, 0.048313044, 0.0018961444, -0.010544702, -0.006205563, 0.029767683, 0.0008878889, -0.0018693962, 0.019472629, -0.03623478, -0.0012608765, -0.025797073, -0.02532155, 0.025701968, 0.00961149, 0.00315925, 0.0024207046, 0.020637657, -0.028626429, -0.0067286375, -0.011911828, -0.005480392, 0.0051504984, 0.023883097, 0.003613968, 0.0018500781, 0.0062650032, -0.0068950704, -0.021992896, -4.411582e-05, 0.010639806, -0.0074775848, 0.015894325, 0.023086596, 0.016964251, 0.021208284, 0.031574667, 0.03704317, -0.025844626, 0.0046452563, 0.013897133, -0.011103441, 0.0025618752, 0.015169155, -0.051594146, -0.015276147, -0.006354164, -0.009498553, -0.0007218277, -0.0012385864, 0.018533474, 0.017784527, -0.039111692, 0.003851729, -0.026391476, -0.021136956, -0.009807643, -0.0047106408, 0.02465582, 0.0026985877, 0.00030481708, -0.03033831, 0.016560057, 0.00590539, 0.0167146, 0.051166177, 0.0032900183, -0.0041667623, -0.0030180793, -0.022016672, 0.03295368, 0.023407575, -0.010437709, -0.011055889, 0.03668653, 0.028103355, 0.009248905, -0.051166177, 0.031693548, 0.01220903, 0.012351686, 0.013992237, 0.026343923, -0.031741098, 0.030219428, 0.04046693, 0.053115815, 0.00053013285, -0.015406916, -0.016310407, 0.013397834, 0.007756954, 0.022515971, 0.018997107, -0.042036153, 0.03568793, -0.012232806, -0.0077272337, -3.3736993e-05, -0.0017460576, 0.0036199118, 0.010509038, 0.018830674, -0.0030433412, -0.025701968, -0.03276347, 0.026248818, 0.00063415326, -0.002765458, -0.011073721, -0.01879501, -0.012280358, 0.028887967, -0.006413604, -0.012173365, 0.014063566, -0.00541798, -0.004939486, -0.006859406, 0.00023831829, -0.0149551695, 0.009724427, 0.0014406833, -0.01629852, 0.018010398, -0.017784527, -0.0004740361, -0.00035181205, 0.006651365, 0.023383798, 0.0323355, -0.024085194, -0.005735985, 0.010818128, -0.0056260205, -0.02465582, 0.039896302, 0.023455126, 0.012042597, -0.010164284, 0.003171138, -0.04883612, 0.020495001, -0.019567734, 0.0024415087, 0.014669856, -0.014883841, 0.04916898, 0.00860695, -0.027247416, 0.053306025, -0.009213241, 0.02136283, 0.014800625, 0.038969036, 0.008707998, 0.017594317, -0.009896804, -0.070567474, -0.02703343, -0.025178894, -0.009225128, -0.02508379, 0.00451746, 0.002032857, 0.0024727148, 0.0048532975, -0.014241886, 0.015145378, -0.008494013, 0.010045404, 0.01192966, 0.030908935, 0.020233465, 0.039325677, -0.021255838, -0.024441835, 0.022920165, -0.017475437, -0.009932468, -0.0018589941, 0.0026302314, -0.061722767, -0.015228595, -0.0025975395, 0.009445057, 0.0014770905, -0.023704777, -0.01684537, 0.000914637, 0.02319359, 0.029577473, 0.049026325, 0.018355152, -0.0060866824, 0.003905225, -0.025440432, 0.002081895, -0.042250138, 0.0073824804, 0.0023731524, -0.0062828357, 0.0149551695, 0.021065628, -0.013350283, -0.042987198, 0.013160073, 0.034380246, -0.0028412445, 0.020720875, -0.008256252, 0.026962101, -0.010087012, -0.0030388832, 0.005504168, -0.026248818, 0.0022334678, -0.030956488, -0.025155118, 0.009510442, 0.026248818, -0.019876823, 0.0027922061, -0.02886419, -0.002028399, 0.024370506, -0.0025767353, -0.01634607, 0.024608267, -0.04826549, 0.0019570705, -0.011685955, -0.015145378, -0.001826302, -0.028983071, 0.060438856, -0.0495494, 0.022908276, 0.0053644837, -0.038588617, -0.02068521, 0.008969535, -0.0069901748, 0.008464293, -0.0005542804, 0.008036323, -0.03618723, -0.0040359935, 0.010235613, 0.0018857423, 0.000901263, -0.017915294, -0.036115903, 0.011275818, -0.024727149, -0.020245353, -0.000919838, 0.017106907, 0.0014622305, -0.010140508, 0.03521241, -0.0062352833, -0.012553783, 0.0440809, 0.016571945, 0.0009933953, 0.00081581756, 0.045602567, 0.008886319, -0.0055814404, -0.019627174, -0.02660546, 0.01861669, -0.00036332858, -0.008731774, -0.007287376, -0.0046809204, 0.023787992, 0.009664986, -0.014574752, -0.043224957, 0.007768842, 0.010051348, -0.008690166, -0.0035218354, 0.016999915, 0.011614627, 0.0021665974, -0.018236272, 0.04027672, 0.009011144, 0.049454298, 0.0023612643, -0.015264259, 0.016607609, -0.03678163, 0.0067821336, 0.0016643273, 0.052687846, -0.004639312, -0.011275818, -0.0011917772, -0.026034834, -0.008464293, 0.00823842, 0.023205478, 0.014622304, -0.023264918, -0.0016227191, 0.0019689586, 0.014075453, -0.004948402, -0.01703558, 0.052640293, 0.0024028725, 0.020055143, -0.00600941, 0.018890115, 0.025583088, -0.0060926266, 0.029387265, -0.0063066115, -0.03278725, -0.0007508048, -0.013207626, -0.022064224, 0.012946089, -0.002955667, 0.018997107, -0.039278124, 0.007786674, -0.0007307437, 0.016322296, -0.030362085, 0.0301481, -0.00029255752, -0.020043256, -0.0010721537, 0.0024459667, 0.0056052166, 0.0051178066, -0.03166977, 0.0063185, 0.027556505, -0.009730371, -0.016857257, -0.0021235032, 0.040062737, -0.0075964653, -0.006853462, 0.028269788, -0.004877073, -0.01464608, 0.0016672993, -0.008761494, 0.0055695525, -0.01666705, -0.021457935, -0.0060569625, 0.02929216, -0.027437624, -0.0151572665, 0.008083875, -0.029006846, 0.0023434323, -0.046363402, -0.02245653, -0.020578219, 0.01012862, 0.008416741, -0.005896474, 0.03276347, 0.02185024, 0.0012527035, -0.012708328, -0.0006114917, -0.025202671, -0.027841818, -0.016013207, 0.056729786, 0.003634772, -0.0060361587, 0.007305208, 0.00592025, 0.00026636667, -0.015704118, 0.0077272337, -0.012048541, 0.013516715, -0.016762154, -0.016227191, -0.020376122, 0.018723682, -0.025583088, 0.025178894, -0.01141253, 0.005236687, -0.033001233, -0.0031384458, 0.015109714, -0.002752084, -0.01055659, 0.015169155, -0.017796414, -0.021065628, -0.039159242, 0.0061461227, 0.03216907, -0.00961149, -0.01861669, 0.004835465, 0.010990504, 0.014562864, 0.011246097, -0.04498439, 0.028626429, 0.028531324, 0.03278725, 0.009777923, 0.028412445, -0.00906464, 0.00860695, -0.020839754, 0.015097826, 0.018771235, 0.042678107, -0.028626429, -0.043843135, -0.018224385, -0.012054484, 0.00640766, 0.005203995, 4.8063022e-05, -0.02563064, -0.023419462, 0.0247747, 0.018783122, -0.010889456, 0.017356556, -0.02062577, -0.008505901, 0.0017698337, -0.0440809, 0.03399983, 0.014860065, -0.012660775, 0.0031116977, -0.004038966, 0.011691899, 0.07128076, 0.017558653, -0.018604802, -0.0049632615, 0.002500949, -0.041655734, -0.013813917, -0.001362668, -0.020352345, 0.020827867, -0.0119593805, -0.011109385, 0.026153715, 0.005269379, -0.018057952, 0.011139105, 0.008119539, -0.006247171, 0.0020298848, -0.014455872, 0.005762733, 0.0015075536, 0.0011984643, -0.011888052, 0.03789911, -0.014170558, 0.02062577, 0.0366152, 0.0005375629, 0.015109714, 0.020352345, 0.012613223, 0.034950875, -0.008315693, -0.0024103026, -0.0037090722, -0.03271592, -0.027794266, -0.015026498, -0.010330717, -0.016060758, -0.0015216706, 0.007061503, 0.010812183, 0.041513078, 0.0074300324, 0.0029452648, 0.018022288, 0.00456204, 0.0571102, -0.030433413, 0.0012787086, 0.0048711295, 0.03882638, -0.008850655, 0.019912487, 0.02850755, 0.0023790963, 0.013706924, 0.018890115, 0.005893502, 0.0141824465, 0.026106162, 0.07689192, -0.018462146, -0.021826463, 0.01201882, 0.00958177, -0.013540491, -0.007513249, 0.055160563, -0.004514488, -0.018640466, 0.0055814404, 0.01575167, 0.002068521, 0.02850755, 0.01238735, -0.023597782, 0.0020016509, 0.008066043, -0.0008225046, -0.017772637, 0.010015684, -0.005236687, -0.04137042, 0.0008343927, -0.017653758, -0.014836289, 0.019436965, 0.028959295, -1.9968213e-05, -0.050120026, -0.0018129279, -0.012280358, 0.007531081, 0.014931394, 0.008077932, 0.027294967, -0.0025975395, -0.02062577, -0.04443754, -0.021232061, -0.008339468, -0.0066929734, 0.015395028, -0.016892921, -0.010859735, 0.010859735, 0.035997022, -0.011721619, -0.005768677, -0.01390902, 0.021600591], "content_hash": "119ceb4bfc9d0c7f00c40ff0b036d7d5"}, {"content": "See in the Dashboard In **Stage 2 (Embedding)** of the pipeline, you can see: - The dimension of the embedding vectors (e.g., \"768-dimensional\" for GPT-2) - A visual showing the flow: Token ID \u2192 Lookup Table \u2192 Vector - An explanation of how the lookup table was created during training\n\nThis is the point where raw token IDs become rich numerical representations that the rest of the model can process.", "source_file": "embeddings_explained.md", "chunk_index": 1, "embedding": [-0.0052048396, -0.0047809435, 0.026281565, -0.005498539, 0.008060084, -0.03330613, 0.015429822, 0.051061325, -0.02945473, 0.040161137, 0.048639063, 0.0144851385, -0.014521473, -0.07194124, 0.02320529, -0.023362737, -0.016798401, 0.022854062, -0.012226378, 0.039143786, -0.00051965134, -0.025554886, -0.005949686, 0.060701936, 0.013080225, 0.018857324, 0.014024909, 0.035558835, 0.030229855, 0.006818673, 0.0064371666, -0.030254077, -0.027129358, -0.023447515, 2.7818189e-05, -0.033185016, -0.008932099, 0.03778732, -0.051594224, 0.031222982, 0.016362393, 0.029478952, -0.08201786, 0.06389932, -0.0075090183, 0.04597457, -0.012704775, -0.06263975, -0.029866515, 0.056099635, -0.013201339, 0.020940473, -0.045611233, -0.006161634, -0.0057286546, 0.0117479805, -0.020153238, -0.03698797, -0.03698797, -0.034541484, -0.018360762, 0.012377769, 0.026935576, 0.016677288, 0.02870383, -0.012813777, -0.068889186, 0.07315237, -0.01578105, 0.024186306, 0.0533867, 0.01897844, -0.0036000898, 0.00373937, 0.0636571, -0.004950502, -0.03432348, 0.019983677, -0.038562443, -0.0585219, 0.024392199, -0.043140523, -0.03640663, -0.01550249, -0.0023253735, -0.036842637, 0.009695112, -0.034614153, -0.0106579615, -0.054016486, -0.04989864, -0.020359129, -0.029575843, 0.0016955849, 0.015308709, 0.054743167, 0.0119659845, -0.042995185, -0.029285172, 0.016919514, 0.002796201, 0.008496091, -0.0045659677, -0.006709671, 0.017609859, -0.03303968, 0.011687424, 0.029236726, -0.030811198, -0.045708124, -0.10028173, 0.005168506, -0.009755668, 0.012716886, 0.03347569, 0.002086175, -0.020577133, 0.026572237, 0.022490721, 0.003044483, -0.042050503, -0.03100498, 0.014727365, -0.0040149027, -0.0490024, 0.03577684, -0.015514601, -0.01100919, -0.020238016, -0.009719335, -0.039676685, -0.012159766, 0.027759146, -0.016313948, -0.009174325, -0.048324168, -0.013383009, 0.06511046, 0.010010006, 0.0006668796, -0.016326059, -0.040572923, 0.029963406, 0.010282511, -0.041808277, 0.030157186, -0.054549385, 0.07116611, -0.02926095, 0.004629552, -0.017137518, -0.009555832, 0.002582739, -0.046289466, -0.032773234, -0.01568416, -0.028340489, 0.020831471, 0.028098263, -0.013334563, -0.025191545, 0.01449725, 0.0687923, 0.0064916676, -0.06816251, 0.018857324, -0.0154661555, -0.040136915, -0.030617418, -0.06065349, 0.008871542, -0.01991101, -0.03807799, -0.0269598, 0.011820649, -0.017815752, -0.05358048, -0.035001714, 0.008617205, -0.0625913, -0.05837656, 0.0017909615, -0.023241622, 0.02540955, -0.02312051, 0.047040366, -0.012214267, 0.014158133, 0.022151604, 0.022502832, -0.0089563215, 0.02422264, 0.054113377, 0.009816225, -0.005519734, 0.0077875787, 0.019487115, 0.022963062, 0.045126777, -0.002513099, 0.029890738, 0.019366002, 0.024283197, -0.003615229, -0.011081858, -0.0076240757, -0.024246862, -0.013613123, -0.023459626, 0.0070608994, -0.014400359, -0.025167324, 0.01050657, -0.0037605648, -0.04505411, 0.0631242, -0.031053424, 0.016495617, -0.0006138925, -0.0078057456, 0.004723415, 0.011118192, 0.04442432, -0.007860247, -0.049753305, 0.009658778, 0.00083113933, -0.04340697, 0.007551408, 0.0032004162, 0.0022345386, 0.012371713, 0.016386617, -0.022987286, 0.010984967, 0.018857324, -0.02376241, -0.059151687, -0.014412471, -0.033063903, -0.019414445, -0.0032700563, 0.0015880968, 0.032119222, 0.0047324984, 0.015805272, -0.0011316515, -0.0277107, 0.051788006, 0.038199104, -0.009186436, -0.011287751, -0.012801665, -0.003581923, 0.0013912879, -0.014509361, -0.02055291, 0.043818757, 0.018881548, -0.047209926, 0.06699982, 0.009028989, -0.08075828, 0.007424239, 0.0011475476, -0.018615099, -0.011336195, -0.014000686, -0.047645934, -0.06767806, 0.016265502, -0.018760435, 0.0049141683, 0.024258975, -0.05643875, -0.0046961643, -0.024343753, -0.007557464, -0.003008149, 0.018639322, 0.009410496, -0.014073354, 0.06758117, 0.059248578, -0.0206498, -0.008974488, 0.01578105, 0.024077304, -0.02119481, -0.02853427, 0.047040366, 0.020807248, 0.05953925, -0.0010938036, -0.021824598, 0.021982046, 0.016931625, 0.019402334, 0.02870383, -0.009997895, 0.023072064, -0.07150523, -0.009216715, 0.0020407573, -0.014715254, 0.03119876, 0.012886444, 0.0125109935, 0.020565022, -0.04679814, -0.013661569, -0.03146521, 0.016846847, -0.0071335677, -0.009355995, 0.025191545, 0.03531661, 0.008514258, 0.024464866, -0.05236935, 0.022030491, -0.021497592, -0.01999579, -0.025942447, 0.027686479, -0.01532082, -0.032603674, 0.030205632, 0.05009242, -0.011796426, 0.03442037, 0.0038059824, -0.027129358, 0.016798401, -0.049389962, -0.019220665, 0.00049240084, 0.022175826, -0.011087913, 0.034565706, -0.05634186, -0.022248495, -0.009386273, -0.043794535, -0.06762961, 0.028146708, -0.015659938, 0.019366002, 0.04028225, -0.023072064, 0.0033608912, -0.033984363, -0.010016061, -0.03274901, 0.030520527, -0.012559439, 0.010724574, -0.039604016, -0.023629185, -0.03635818, -0.027686479, 0.0006555252, 0.034735266, 0.019378113, -0.04093626, -0.010482348, -0.020383352, 0.018820992, -0.0072001796, -0.0037878153, 0.011711647, 0.013080225, -0.0066370033, -0.004644691, -0.017488746, 0.0013398147, 0.033814806, 0.018881548, -0.04636213, 0.005089782, 0.029309394, 0.0068610627, -0.02790448, -0.020565022, -0.0029309394, 0.049099293, 0.02918828, 0.007321293, -0.035365056, 0.0024056109, -0.028025595, -0.031731658, 0.009701167, 0.0016547091, 0.007969249, 0.010748796, -0.0069095083, 0.018288093, 0.027105134, 0.0006392506, -0.040645592, -0.04502989, -0.016568286, -0.045635454, 0.038707778, 0.0016425978, 0.03357258, -0.01578105, 0.02166715, 0.031658992, -0.008181197, 0.022841949, 0.011172693, -0.05624497, -0.0069882316, 0.016677288, -0.011669257, 0.041590273, -0.0062736636, 0.081775635, -0.046216797, 0.0075392965, -0.050867543, 0.006316053, -0.024925096, -0.0048112217, 0.049026623, -0.00858087, -0.011978095, 0.0103854565, -0.020141125, 0.04624102, 0.040209584, -0.014024909, -0.021328034, -0.03439615, 0.055469844, 2.1833494e-05, -0.015369265, 0.020674024, -0.0066915043, -0.0030051214, -0.023568628, -0.030593194, -0.04936574, -0.020783026, -0.018336538, -0.0405487, 0.020783026, -0.03534083, 0.03897423, 0.038005322, -0.028558493, 0.0073818495, -0.030714307, -0.08221164, 0.019523447, 0.033621024, 0.036212847, 0.0089018205, 0.007230458, 0.016544063, -0.03781154, 0.021848822, 0.009192492, 0.00033268283, -0.012971223, -0.029793847, 0.039797798, -0.022309052, -0.01770675, -0.009307549, 0.00940444, -0.016762067, 0.001175555, -0.01643506, 0.0433343, -0.02991496, -0.034008585, 0.023798743, 0.012056819, -0.0010574696, -0.061137944, -0.0029097446, 0.02027435, 0.12256656, 0.00073273486, -0.01068824, 0.044714995, -0.003173166, -0.038514, -0.048324168, 0.008247809, -0.012317212, 0.010797242, 0.021328034, -0.031271428, -0.05580896, 0.0058558234, 0.001889366, 0.026596459, 0.016362393, 0.005858851, 0.028219376, 0.04706459, 0.009150103, 0.00784208, 0.022103159, 0.0046628583, 0.03396014, -0.022442276, 0.026451124, -0.011784314, 0.007696744, 0.0008031319, 0.010100841, 0.014388248, 0.0087928185, 0.017089073, 0.0057135154, 0.007745189, 0.0020680078, 0.022732947, -0.019499226, -0.019402334, -0.016204946, -0.02863116, -0.031319875, -0.005026198, -0.012044708, 0.021340147, -0.048857067, 0.0040482087, 0.0045296336, -0.078336015, 0.019668784, -0.0036909247, 0.022623945, 0.008538481, 0.039579794, 0.008950265, 0.040887818, 0.028001372, 0.0003502821, -0.008950265, 0.03044786, 0.0018484902, 0.0033457521, 0.0008122154, 0.04192939, 0.026063561, -0.028267821, 0.034347706, -0.00060518755, -0.02320529, -0.0041965726, 0.015199707, 0.016943736, 0.0010786644, 0.0034547541, -0.008701984, 0.024101527, 0.061137944, -0.016737845, -0.046628583, 0.05479161, 0.025021987, -0.0438672, 0.02797715, -0.054597832, -0.0548885, 0.016677288, 0.029478952, 0.05144889, -0.014388248, 0.007896581, -0.0033003348, 0.02991496, -0.025070433, 0.014872701, -0.01486059, -0.010125063, 0.005680209, -0.020116903, -0.020480242, 0.023435405, 0.019874677, -0.010276455, -0.022829838, -0.007406072, 0.055082284, -0.0049626133, -0.02211527, 0.017137518, -0.011645034, -0.013334563, 0.025021987, -0.016083833, -0.024900874, -0.018360762, 0.0010998592, -0.004517522, -0.013225562, -0.0023934997, -0.013298229, 0.024464866, -0.022103159, 0.040960483, -0.013128671, 0.01211132, -0.009458941, -0.051594224, -0.011669257, 0.012959112, -0.020661913, -0.006552224, -0.011808537, 0.008374978, 0.0029233699, 0.025530662, 0.015332932, -0.020395463, -0.0044963276, -0.039943133, -0.01367368, 0.0423654, -0.019317556, -0.027080912, -0.029042946, 0.008665649, -0.025021987, 0.010585294, 0.04478766, -0.0070124543, 0.012099208, 0.011881205, -0.0026402678, -0.0018197258, 0.0126199955, -0.04175983, 0.017464524, -0.011614756, 0.004986836, -0.02550644, -0.0106034605, 0.021267477, 0.00334878, 0.029406285, -0.031731658, -0.030690085, -0.029212505, -0.044521213, 0.033063903, -0.0070972336, -0.003064164, -0.041348048, -0.024440644, -0.023617074, -0.00044509102, 0.0038392884, 0.0038211215, -0.0055469847, -0.022732947, -0.023592852, -0.0118569825, 0.048687506, 0.011039468, -0.004665886, 0.0040179305, -0.021885155, -0.04733104, 0.053144474, -0.0029142865, 0.009071379, 0.0028915778, -0.035461944, -0.03340302, 0.01321345, -0.0015441934, 0.030883865, -0.01926911, 0.025482217, -0.0066551706, 0.004335853, -0.0007319779, -0.011414919, 0.032531004, -0.01285011, 0.02293884, 0.017028516, -0.017282853, 0.0020089652, 0.015635714, -0.01385535, -0.009198547, 0.0067460053, -0.008913931, 0.045272116, -0.008780707, -0.024016747, -0.00958611, -0.022914618, -0.042995185, -0.052611575, -0.010082674, -0.016313948, 0.014969592, 0.011196915, 0.01458203, -0.017924754, -0.031344097, -0.001372364, 0.012426214, -0.0074908515, -0.014981703, 0.019765675, -0.034057032, -0.0017364605, -0.0069095083, 0.0016425978, -0.028195154, 0.004705248, -0.02586978, 0.0051382277, 0.009168269, -0.045102555, -0.033257686, -0.010615572, 0.018578764, 0.007109345, 0.023713965, 0.0017001266, -0.005577263, -0.050528426, 0.026378455, 0.017864197, -0.012632106, 0.02358074, -0.031222982, 0.013334563, -0.040766705, 0.019196441, -0.0086838165, -0.008659594, 0.008641426, 0.0104944585, -6.3454327e-06, -0.023277957, 0.01871199, -0.05982992, -0.007006399, 0.008054028, 0.0071880682, -0.03543772, 0.004117849, -0.0018454624, 0.05144889, -0.017839974, 0.008108528, 0.002419236, 0.018397095, 0.016495617, 0.016858958, 0.04791238, 0.021170588, -0.029793847, 0.012595773, 0.022527056, 0.026596459, 0.0052684243, -0.016338171, -0.005444038, 0.01018562, -0.001924186, 0.065934025, -0.040694036, 0.02853427, -0.009519498, -0.0018727129, 0.009792002, -0.037641983, 0.0042086835, 0.0004560669, -0.027783368, 0.013649458, 0.008181197, -0.01670151, -0.0108214645, 0.0076725213, -0.024561757, -0.039797798, 0.00196052, 0.0041239043, 0.003578895, -0.01467892, -0.0074545173, 0.0124019915, -0.08817041, -0.012898556, 0.027274692, -0.0073273485, 0.024743427, -0.004896001, 0.038465552, -0.015732605, -0.0021951767, -0.025651775, -0.014436694, -0.01206893, 0.019959455, -0.02494932, 0.0152481515, -0.031780105, 0.028824942, 0.008435534, -0.0075756307, -0.0074666287, -0.035849508, -0.0054773446, 0.04842106, 0.00812064, -0.065352686, -0.003918012, -0.00039853813, 0.007103289, 0.041130044, 0.0056438753, -0.026911354, 0.022890395, -0.0042147394, 0.037036415, 0.014291357, -0.0006343304, -0.0071154004, -0.025918225, 0.04277718, 0.03146521, 0.018070089, -0.046289466, -0.0075392965, 0.004350992, 0.050334647, 0.0024843346, 0.004420632, -0.0030747615, 0.012995446, 0.009689056, 0.027177801, 0.02761381, -0.017004292, -0.054064933, 0.024016747, 0.01569627, 0.0028658411, -0.011112136, 0.030011851, -0.007848135, -0.0055802907, 0.0035153106, 0.023350624, -0.0059920754, 0.011318029, 0.0063039423, 0.028219376, 0.016822623, -0.01050657, -0.042341176, -0.013007558, -0.008005583, 0.014158133, 0.0024449727, -0.02274506, 0.017537192, -0.0038090101, 0.011729813, -0.035147052, 0.003008149, 0.022914618, 0.0073636826, -0.029624289, 0.028582714, -0.00046060863, 0.017391855, -0.02330218, -0.024997765, -0.01211132, -0.012032596, 0.017331298, -0.014703142, -0.03175588, 0.02146126, -0.0021815514, -0.00084173674, 0.007073011, 0.0224665, 0.04541745, -0.013419342, 0.07053633, -0.0015063455, -0.041711386, -0.016059611, -0.046459023, -0.008399201, 0.012607885, -0.016640954, -0.008205419, -0.019705119, -0.0067217825, -0.0015593325, -0.024852429, 0.0055802907, -0.028388934, 0.022345386, 0.034081254, 0.041784056, 0.017646194, -0.061719287, -0.00037223386, -0.019196441, -0.007127512, 0.031683214, -0.041493382, 0.0016501674, 0.014606252, 0.022175826, -0.012668441, -0.016943736, -0.04488455, 0.036261294, 0.025772888, 0.009525553, -0.028219376, 0.028970277, 0.0037545091, 0.0056590145, -0.06607936, 0.02166715, -0.01247466, -0.0039482904, 0.0070972336, -0.015550935, 0.007769412, 0.03580106, -0.015865829, -0.013443565, -0.006340276, 0.0017758223, 0.029236726, 0.017827863, -0.022769282, 0.02127959, 0.02072247, 0.012323268, 0.007303126, -0.038707778, -0.009901004, 0.048396833, 0.018942105, 0.030350968, 0.0058739902, 0.0080843065, 0.011941762, -0.026063561, 0.011100025, 0.022163715, -0.04597457, 0.019475002, -0.011390696, 0.012323268, -0.003312446, 0.030350968, 0.027177801, 0.040330697, -0.016071722, 0.050625317, 0.028800718, -0.021267477, -0.024186306, 0.033136573, -0.010797242, -0.0061858566, 0.028824942, -0.025433771, -0.019886788, 0.0063342205, -0.02945473, 0.033063903, -0.0008886681, 0.0128379995, 0.006231274, 0.0179732, -0.008835208, -0.015526712, -0.012498883, 0.018009532, -0.0043237414, 0.031247206, -0.005649931, -0.010930466, 0.0065946137, 0.017948976, -0.030835422, -0.011336195, 0.013140783, -0.0056741536, 0.025433771, -0.013940129, 0.015926385, 0.012825888, 0.017537192, 0.042195838, -0.027468475, -0.009253048, 0.0026402678, -0.016834734, 0.013237673, 0.0038029545, -0.023544407, -0.0120749865, -0.016713621, -0.0025978782, -0.022284828, -3.8770417e-05, 0.032991234, 0.025021987, -0.027589587, -0.005952714, -0.020201681, -0.031610545, 4.1254185e-05, 0.0029279117, 0.029478952, 0.006443222, 0.0010809354, -0.013588901, 0.022139493, -0.0010468722, 0.010330956, 0.038683556, 0.0041208765, 0.008108528, -0.003330613, -0.015720494, 0.030423636, 0.026523791, -0.008568759, 0.003581923, 0.046628583, 0.005080699, 0.0115844775, -0.047549043, 0.030060297, 0.019838342, 0.02981807, 0.005698376, 0.011057635, -0.019814119, 0.016180724, 0.045538563, 0.054064933, -0.013383009, -0.0058739902, -0.01233538, 0.00669756, 0.010700352, 0.015817383, 0.036648855, -0.02045602, 0.026669126, -0.021642929, -0.005289619, 0.012765331, 0.007666466, 0.014073354, -0.0059284912, 0.023532296, -0.0076301317, -0.028582714, -0.040766705, 0.024585979, -0.012898556, -0.020964695, -0.0029142865, -0.026063561, -4.6363646e-05, 0.03633396, -0.019111663, -0.0074545173, 0.038295995, 0.001978687, -0.0005404677, 0.007321293, 0.004614413, -0.0004151912, 0.0053683426, 0.0053350367, -0.038174883, 0.016895292, -0.020322796, -0.006040521, -0.02092836, -0.0056771813, 0.01569627, 0.033451468, -0.0046719415, -0.019099552, -0.0046779974, -0.0005185159, -0.03485638, 0.014436694, 0.019426558, 0.010851743, 0.004523578, -0.012595773, -0.038853116, 0.025312658, -0.011772203, -0.009640611, 0.022914618, -0.010288566, 0.027807591, 0.0047991103, -0.028679606, 0.038005322, -0.019487115, 0.016471395, 0.023847189, 0.048057716, 0.010028173, 0.0034698932, 0.002670546, -0.033693694, -0.032046553, 0.0011528463, -0.013540456, -0.028655384, -0.014267135, 0.021340147, 0.017948976, 0.0045356895, -0.008211475, 0.0040512364, -0.0006350874, 0.010119008, 0.019680895, 0.048106164, 0.025167324, 0.042123172, -0.020238016, -0.0019680895, 0.011620811, -0.013370898, -0.004008847, 0.0002475251, -0.008296254, -0.05697165, 0.004547801, 0.019668784, 0.015066482, -0.008005583, -0.0016834735, -0.007914748, -0.0011384641, 0.031150315, 0.005625708, 0.056487195, 0.011142414, 0.0019332694, -0.007908692, -0.016386617, 0.0081388075, -0.022006268, 0.009798058, 0.009028989, -0.002190635, 0.013467788, 0.022260606, -0.036648855, -0.05367737, 0.023520183, 0.026451124, -0.009246993, 0.009991839, -0.00025793328, 0.04083937, 0.0034577819, 0.0014889354, 0.020577133, -0.0302783, 0.0200079, -0.029042946, -0.019971566, 0.007085122, 0.018457651, -0.020201681, 0.003224639, -0.022829838, 0.0010271913, 0.029115614, 0.005822517, 0.012141598, 0.031707436, -0.031513654, 0.0047627767, 0.0028370768, -0.011439142, 0.0075635193, -0.013274007, 0.029478952, -0.024900874, 0.02981807, 0.0075756307, -0.0559543, -0.011075802, 0.007345516, -0.0095134415, 0.022563389, 0.02037124, 0.006770228, -0.027541142, -0.020395463, 0.015659938, 0.006179801, 0.038707778, -0.014787922, -0.035704173, 0.02376241, -0.036551964, -0.025264213, 0.0053168694, 0.0012694177, -0.0023586797, 0.0020998002, 0.032167666, -0.00020702787, -0.0017667388, 0.029672734, 0.008368922, -0.0115299765, -0.0067278384, 0.03778732, -0.007502963, -0.011548144, -0.010785131, -0.027323138, 0.019571893, 0.0051927282, -0.0042995187, 1.7575607e-05, 0.015272374, 0.013794794, 0.004235934, -0.009792002, -0.023290068, 0.020165348, 0.0064553334, 0.00016993696, -0.003578895, 0.012159766, -0.0006199482, 0.022854062, -0.004971697, 0.020783026, 0.010972856, 0.0585219, -0.0060980497, 0.018082201, 0.020589244, -0.015429822, 0.019099552, -0.01022801, 0.03577684, -0.00039740268, 0.0024510284, -0.016568286, -0.019935234, -0.02467076, 0.012359602, 0.03393592, 0.019935234, -0.023241622, 0.010300677, 0.007969249, -0.0057468214, -0.026984021, -0.0010037257, 0.059103243, 0.007357627, 0.008853375, -0.0025403495, -0.003969485, 0.037012193, 0.008205419, 0.025288437, -0.00070813374, -0.038005322, 0.0035334777, -0.0104944585, -0.021982046, 0.0029339674, -0.0011785828, 0.023108399, -0.04488455, -0.022309052, 0.0054016486, 0.011160581, -0.023386959, 0.027226247, 0.013685792, -0.028679606, 0.0075271856, 0.010978912, -0.008508203, -0.009422607, -0.030254077, -0.004148127, 0.0033245573, -0.0087928185, -0.025433771, 0.0059375744, 0.032700565, 0.0068005063, -0.01992312, 0.028025595, 0.026087783, -0.028195154, 0.009785946, -0.021945711, -0.009604277, -0.0062009958, -0.019608228, 0.009979728, 0.029139835, -0.04347964, -0.008586926, -0.00033097967, -0.017597748, 0.009028989, -0.03938601, -0.011445197, -0.016846847, 0.0069337306, 0.03383903, -0.0061888844, 0.048081942, 0.011124248, -0.0028643273, -0.023374848, -0.003972513, -0.026911354, -0.016289726, -0.018421318, 0.038950004, -0.0019514364, -0.012983335, 0.0071154004, -0.011760091, 0.0055893743, -0.017852087, 0.009186436, -0.0029430508, 0.023072064, -0.019608228, -0.007345516, -0.020347018, 0.019680895, -0.038950004, 0.025118878, -0.010258288, 0.011687424, -0.021509705, -0.0055863466, 0.011196915, -0.0065280017, -0.01926911, 0.041614495, -0.004260157, -0.023786632, -0.0354135, 0.004235934, 0.010161398, -0.008035861, -0.0057922387, 0.013588901, 0.018578764, -0.0064371666, -0.001175555, -0.04093626, 0.024573868, 0.026984021, 0.036551964, 0.008253865, 0.023750298, -0.0072546806, 0.020783026, -0.025094654, 0.02935784, 0.00936205, 0.031586323, -0.021703485, -0.037036415, -0.029236726, 0.02293884, -0.00341842, 0.018082201, -0.014824255, -0.030617418, -0.030617418, 0.025433771, 0.016725734, -0.025264213, 0.013007558, -0.019693006, -0.003472921, 0.017839974, -0.027952926, 0.03240989, -0.002814368, -5.3223573e-07, 0.012141598, -0.0013239187, 0.01614439, 0.08884864, -0.0064613894, -0.0005836142, 0.00037071994, 0.0075937975, -0.009471052, -0.008435534, 0.0069337306, -0.008247809, 0.008041916, -0.0029475924, -0.0114754755, 0.0114209745, 0.023544407, -0.005610569, -0.012389881, 0.000729707, -0.0075211297, -0.024997765, -0.015623603, 0.015441933, -0.0060435487, 0.004559912, -0.008968432, 0.039143786, -0.0014003714, 0.006764172, 0.03696375, -0.010555016, 0.020201681, 0.026039338, 0.008205419, 0.023653409, -0.0072546806, -0.011178749, -0.034735266, -0.030326745, -0.018215425, -0.011015246, -0.0071517345, -0.023338513, -0.0016289726, 0.008980544, 0.03202233, 0.033717915, 0.012371713, 0.0057982947, 0.0054561496, 0.032676343, 0.052272458, -0.026766017, 0.010488403, 0.011590533, 0.03972513, -0.0074545173, 0.01467892, 0.045587007, 0.008471868, 0.0007013211, -0.00031375888, -0.00033192587, 0.011681369, 0.011826704, 0.05236935, -0.024271086, -0.026862908, 0.018518208, -0.017936865, -0.008919987, 0.0050534485, 0.04393987, -0.008883653, -0.010536849, -0.0007039705, 0.014267135, -0.011015246, 0.030108742, -0.002367763, -0.012583662, 0.014351915, 0.008538481, -0.0037181752, 0.0068489513, -0.018506097, -0.025894003, -0.03182855, -0.015369265, 0.0036969804, -0.015284486, 0.0374482, 0.015163372, -0.0068852855, -0.057116985, -0.010209843, -0.021982046, -0.0016410839, 0.024089416, 0.016737845, 0.028970277, 0.008998711, -0.014230801, -0.04449699, -0.017500857, -0.0033911697, -0.007999527, 0.012716886, 0.006733894, -0.009325717, -0.00063811516, 0.028873388, -0.022866173, -0.02018957, -0.0064189998, 0.016495617], "content_hash": "1495cb0f87b7e94b260b27c31df51009"}, {"content": "# Experiment: Exploring Alternative Predictions with Beam Search\n\n## Goal\n\nLearn how beam search reveals multiple possible continuations of a prompt, and see how ablating attention heads can redirect the model's generation from one path to another.\n\n## Prerequisites\n\n- Complete \"Your First Analysis\"\n- Complete \"Your First Ablation\"\n\n## Steps\n\n### Step 1: Generate Multiple Beams\n\n1. Select **GPT-2 (124M)** and enter the prompt: `Once upon a time there was a`\n2. Set **Number of Generation Choices (Beams)** to **3**.\n3. Set **Number of New Tokens** to **8**.\n4. Click **Analyze**.\n\n### Step 2: Compare the Beams\n\nYou should see 3 different generated sequences. Look at how they differ:\n- **Beam 1**: The model's top-ranked overall sequence\n- **Beam 2**: The second-best sequence\n- **Beam 3**: The third-best sequence\n\nNotice how they might start the same but diverge at some point. For example:\n- Beam 1: \"Once upon a time there was a young man who lived\"\n- Beam 2: \"Once upon a time there was a little girl who loved\"\n- Beam 3: \"Once upon a time there was a king who ruled\"\n\nThe beams share a common prefix because the early tokens were confident, but as generation continues, different paths emerge.\n\n### Step 3: Select a Beam for Comparison\n\n1. Click **\"Select for Comparison\"** on Beam 1 (the top-ranked sequence).\n2. This stores it as the baseline for ablation comparison.\n\n### Step 4: Investigate What Drives the Divergence\n\n1. Look at **Stage 5 (Output)** to see the top-5 predictions for the immediate next token.\n2. Note the top prediction and its probability. Are the alternatives close in probability? If so, the model was uncertain, which explains why beams diverge early.\n\n### Step 5: Ablate a Head and Re-Generate", "source_file": "experiment_beam_search.md", "chunk_index": 0, "embedding": [0.04827575, 0.04152533, 0.029295666, -0.020938, -0.025131445, -0.011009619, -0.0128871715, 0.022793636, 0.031326637, 0.07007582, 0.03261243, -0.044184588, -0.029821673, -0.014172966, 0.0050664702, -0.001339674, 0.00023309597, -0.015064255, 0.0061696237, 0.10093489, -0.02730853, -0.02976323, 0.0045477687, 0.012280802, 0.023334254, -0.039771974, 0.004788855, 0.022486798, 0.0046865763, -0.012470749, 0.003638215, -0.022253018, -0.020499662, -0.014728195, -0.048188083, 0.0014976588, -0.024736939, 0.022486798, -0.01203241, -0.0074079316, 0.0028711215, -0.03854462, -0.02966095, 0.025014555, -0.0016035907, 0.05362349, 0.013062507, -0.016539996, -0.009139371, 0.03305077, 0.018614803, 0.0732903, -0.026519518, 0.0035797697, -0.015750987, -0.010213302, -0.031531196, -0.040707096, 0.012083549, -0.040794764, -0.0011871685, 0.017708901, 0.030303847, 0.047954302, 0.0052308473, -0.012441526, -0.048071194, 0.036732823, 0.015108089, 0.030654518, 0.026022734, 0.0007944897, -0.021069502, 0.013975713, -0.030946745, -0.009424292, 0.04491515, 0.0018738998, -0.03635293, -0.03807706, 0.034774907, 0.019579148, 0.033752114, -0.027483866, 0.0074481126, -0.010074494, -0.005409836, -0.021011056, -0.06247794, -0.040473316, -0.022267628, 0.02114256, -0.03459957, 0.044038475, 0.03357678, 0.026417239, -0.009490043, -0.04164222, 0.030946745, 0.034102786, 0.020543495, -0.0042153616, -0.036031477, -0.013654265, 0.07060183, 0.012120077, 0.0033752115, 0.035359357, -0.033313777, -0.033869006, -0.041174658, -0.015239591, -0.005482892, 0.0035578527, -0.0040290672, 0.01725595, -0.012960227, 0.017928071, 0.04959077, -0.019959042, -0.049240097, 0.000374643, -0.01892164, -0.013194009, 0.00043719765, -0.031092858, 0.06428974, -0.06247794, 0.006377835, -0.004134999, 0.017095227, 0.009066314, 0.0066700606, -0.053711157, -0.052834477, 0.031005189, -0.03278777, 0.012025104, -0.05485084, 0.040122643, -0.0044527953, -0.0012730099, -0.013062507, 0.0012173044, -0.061601263, 0.01035211, 0.006933064, 0.045061264, -0.020850332, -0.0009734782, 0.04152533, -0.023202753, -0.028389765, -0.029003441, 0.0053258208, -0.047311407, -0.022150738, 0.024517769, -0.029734006, -0.049999885, 0.027717646, 0.0217124, -0.02776148, -0.0021387297, -0.034541126, 0.051344126, -0.032992326, 0.0443307, -0.02104028, -0.034219675, -0.020090545, -0.006253639, -0.006304778, 0.0011615987, 0.021653954, 0.0048911343, -0.086966485, -0.07498522, -0.040473316, -0.061601263, 0.0067796456, 0.018234909, -0.023246586, -0.00850378, -0.005804341, -0.029164165, -0.0033240719, -0.00079494633, -0.013347427, -0.014662445, -0.0029332195, 0.033518333, 0.04845109, 0.010257136, -0.060841475, -0.020996446, 0.059497233, 0.047340628, 0.033109218, -0.0026756953, 0.04894787, -0.0036546527, 0.022852082, 0.012054327, 0.016949113, 0.037901726, -0.003581596, 0.021244837, -0.05888356, -0.031823423, 0.046873067, -0.05701331, 0.04231434, -0.0018848584, -0.032261763, -0.028404377, 0.006180582, 0.023261197, 0.007890104, 0.045207378, 0.016174715, -0.012514583, 0.018644026, 0.053477377, 0.018234909, 0.028492045, 0.052366916, -0.03471646, 0.03582692, 0.001366157, 0.026431851, -0.007678241, -0.01881936, -0.0018291527, 0.015750987, -0.05184091, -0.009059009, 0.007524822, 0.0128871715, -0.060198575, 0.0009497348, -0.0054354053, 0.048217308, -0.010848894, -0.013727321, 0.032203317, -0.047077626, -0.052337695, 0.00036414113, 0.07498522, 0.024707716, -0.0016839529, -0.019842152, 0.014684361, -0.0107320035, -0.017650457, -0.009110149, -0.01304059, -0.022267628, -0.024590826, -0.009358541, 0.018512523, -0.017986517, 0.002750578, -0.024313211, 0.0070134266, -0.011287233, -0.038456954, -0.043512467, -0.041233104, -0.025613617, -0.011849768, 0.0068527022, -0.005614394, -0.025467504, -0.045937944, -0.0013780287, -0.0010711913, -0.0028802536, -0.01275567, -0.037434164, -0.01035211, 0.03404434, -0.020178212, 0.00027487517, 0.020879555, -0.04164222, 0.016379273, -0.0077951313, -0.011973964, 0.010673558, -0.051227234, 0.026899412, -0.025555173, 0.011316456, 0.024780773, 0.004208056, 0.006096567, -0.04812964, 0.036206815, 0.0166715, -0.029748617, 0.038573846, -0.02194618, 0.016306216, 0.011046147, -0.024342434, 0.016130881, -0.019783707, 0.005135874, -0.0019012961, 0.015371093, -0.045032043, -0.05742243, -0.0036710904, -0.029412556, 0.022326075, -0.02786376, -0.0008200595, -0.032174096, -0.029018052, -0.03658671, -0.0006232635, 0.004018109, 0.008993259, 0.03471646, 0.010191386, -0.018527135, 0.012280802, -0.009139371, 0.004478365, 0.013113646, 0.025482116, 0.017168283, -0.048655644, -0.036031477, 0.009416986, 0.04827575, 0.037317272, 0.026607186, -0.06727045, 0.010008744, 0.0040655956, -0.022501409, -0.035184022, 0.03807706, 0.008175025, -0.026519518, 0.011316456, 0.0029478308, -0.03737572, -0.01947687, -0.01924309, -0.012266191, 0.026271127, 0.028404377, 0.002390775, -0.025613617, 0.029383335, -0.006312084, -0.013033284, 0.06025702, 0.023100473, 0.0038135506, 0.0053477376, -0.028068317, -0.024094041, 0.0007474596, 0.017869625, -0.0017232208, -0.010417861, 0.011367596, -0.0057860767, 0.029251833, -0.0714785, 0.012762975, -0.021084113, 0.0033806907, -0.0008054482, 0.078550376, -0.03404434, -0.02730853, -0.05385727, -0.048509534, 0.012952922, 0.057276316, -0.027001692, 0.008839839, -0.036440596, 0.013150174, 0.0145236375, -0.029748617, 0.007926633, 0.014501721, -0.024210932, 0.05385727, -0.022267628, 0.061192144, 0.048772536, 0.065692425, -0.0041240407, -0.018147241, -0.06417285, -0.026694855, -2.6939593e-05, 0.015312647, 0.039216742, 0.02989473, 0.04061943, 0.022457575, 0.036440596, 0.012346553, 0.048217308, -0.04155455, 0.022048458, 0.033226106, 0.055435292, 0.038603067, 0.020528883, 0.03594381, 0.027250083, -0.08123886, -0.005154138, -0.02554056, -0.041349992, 0.0025788953, 0.015312647, -0.0056691864, 0.050964233, -0.06212727, -0.0120689375, 0.047340628, 0.054354053, -0.022340685, -0.018439466, -0.018965473, 0.0078097424, 0.07545278, -0.034570348, -0.054704726, 0.017781958, -0.004335905, -0.045821052, -0.011857074, -0.031297415, 0.0075832675, -0.056691863, -0.037697166, -0.00929279, -0.016364662, -0.011075369, -0.015707154, -0.0059029674, 0.056604195, 0.00873756, 0.0028857328, 0.05978946, -0.017489731, 0.11057836, 0.0058700917, 0.026563352, -0.042928014, -0.018220298, 0.018790139, -0.008291916, 0.036528263, 0.013639653, 0.010673558, 0.014596694, -0.004376086, 0.044739816, -0.0028528573, -0.0035962074, -0.039362855, -0.030742185, 0.0051577906, 0.035300914, 0.027586143, 0.01700756, -0.037697166, 0.0030263665, 0.014596694, 0.064523526, 0.060958363, -0.020777276, 0.06990048, 0.008861757, 0.008065441, 0.020894166, 0.012646085, -0.00962885, -0.0120689375, 0.007992384, -0.01858558, 0.017781958, -0.059701793, -0.015298036, 0.015298036, 0.01927231, 0.0345119, 0.014509026, -0.020338936, -0.002511318, 0.020338936, 0.028638158, -0.00895673, 0.025905844, 0.024678495, -0.012112772, 0.02855049, -0.00638514, -0.03781406, 0.010995007, 0.013588513, -0.017489731, 0.031589642, 0.012025104, 0.022939749, 0.030099288, 0.03649904, -0.0032437097, 0.011572153, 0.0028272874, 0.0217124, 0.0033350303, -0.01971065, -0.04713607, -0.053243596, -0.020324325, -0.002303107, 0.029251833, -0.029076496, 0.0057678125, 0.03871996, -0.04623017, -0.04038565, -0.007758603, 0.016803, -0.031735756, -0.031618863, -0.0011049799, 0.008372278, -0.002966095, 0.007846271, 0.016627666, -0.011111897, 0.057247095, 0.018746305, 0.0141948825, 0.014808558, 0.03582692, -0.02809754, 0.014034159, -0.03804784, 0.003329551, 0.022442965, -0.025496727, -0.051694795, 0.0039340938, -0.039041407, 0.0037879809, -0.026840968, -0.00021426109, -0.023436533, -0.022150738, 0.019111587, 0.0034099133, 0.0035907282, 0.0039413995, 0.00528564, -0.005168749, -0.0039085243, 0.004631784, 0.012836032, 0.039655082, 0.00023789032, 0.023100473, 0.017592011, -0.0062755556, -0.0039815805, -0.018950863, -0.03123897, -0.027819924, 0.027819924, -0.0077951313, -0.032407876, 0.0023268503, 0.040151868, -0.018833973, 0.01746051, 0.023085862, -0.02117178, 0.00747003, 0.031589642, -0.039129075, -0.0074115843, 0.020455828, -0.0035779434, -0.00024062993, -0.018600192, 0.027337752, -0.028565101, -0.026753299, 0.01870247, -0.018556358, 0.028725825, -0.027030915, -0.009102843, 0.039012186, -0.011491791, -0.012083549, 0.007316611, -0.039713528, -9.5030555e-05, -0.019184643, -0.004357822, -0.014026853, 0.0017926245, 0.030654518, -0.022048458, 0.019783707, 0.009022481, -9.046453e-05, -0.0020254923, 0.011484486, -0.009285484, -0.026490295, -0.0060015935, 0.026402628, 0.036119144, -0.026285738, 0.019330757, 0.027293919, 0.011637905, -0.006118484, 0.029076496, 0.02105489, 0.01277028, -0.007802437, 0.0021533412, -0.0022245713, -0.0020145336, 0.0075175166, -0.019885987, -0.026957858, 0.006253639, 0.010680864, 0.0158971, -0.013449706, -0.045733385, -0.045879498, -0.03693738, 0.08690804, -0.00912476, 0.036002256, -0.018307965, -0.0077951313, -0.024532381, -0.00030181478, -0.037229605, -0.022574466, 0.024897663, -0.04041487, -0.03459957, 0.0040071504, 0.0235242, 0.021639343, 0.030537628, 0.00489844, -0.00828461, -0.04558727, 0.0044491426, -0.002312239, -0.010191386, -0.022194572, 0.006560476, 0.0046537006, -0.0476913, 0.0104324715, 0.017767347, 0.00032921098, 0.017387453, -0.054792393, -0.022223795, 0.0080581345, -0.017387453, -0.015020422, -0.0192577, 0.025277557, 0.0070280377, -0.002648299, 0.0020309715, 0.0071814563, -0.0017232208, -0.002903997, -0.021639343, -0.017986517, 0.022004625, -0.0046756174, -0.07714769, -0.036908157, -0.008525697, 0.0067613814, -0.0029113025, -0.014647833, -0.032846212, -0.013975713, 0.055669073, 0.009957604, -0.047194514, -0.015970156, -0.033518333, -0.008182331, 0.042898793, -0.01791346, 0.013676181, -0.0062755556, -0.00053285604, 0.00056162203, -0.018264132, -0.03305077, 0.025175279, -0.034132008, -0.0075613502, 0.012733752, -0.03950897, -0.0040911655, 0.006312084, -0.022501409, 0.00856953, 0.026972469, 0.0011214176, 0.010819672, -0.014874308, 0.0098041855, 0.021814678, -0.02809754, -0.0021277713, -0.0044710594, -0.016978336, 0.042665012, -0.00794855, 0.019739874, 0.006998815, 0.020134378, 0.009826102, -0.0123684695, 0.03939208, -0.023991764, 0.0125876395, -0.012762975, 0.014012242, -0.009431598, -0.018527135, 0.044739816, 0.005194319, 0.01354468, -0.011703655, 0.00058582204, 0.038661513, 0.012828725, -0.0057020616, -0.016320828, 0.0106224185, -0.01645233, -0.022369908, 0.01393188, 0.0006995162, 0.026183458, 0.025029166, -0.00019462715, 0.013698098, 0.0070097735, 0.0038793015, 0.025175279, -0.010658947, 0.05762699, 0.0012008667, -0.044038475, 0.007627101, 0.018176464, 0.016189326, -0.04208056, -0.010469, -0.0082773045, 0.0072618187, -0.019637594, -0.014326384, -0.053565044, -0.01282142, -0.011382206, -0.0016118096, 0.02799526, -0.03112208, -0.003912177, 0.014151049, 0.0014903531, -0.10549362, -0.0024017333, 0.0028930383, 0.031531196, -0.020938, 0.005530379, 0.026373405, -0.004514893, -0.0036857016, 0.028579712, 0.011871685, -0.033664446, 0.0074955993, -0.010702781, 0.012492666, -0.04021031, 0.014326384, -0.017416677, 0.00046893157, -0.026008124, -0.02742542, 0.015064255, -0.026840968, -0.007802437, -0.027410809, 0.022501409, -0.019316146, 0.011331067, -0.006144054, -0.0045952555, -0.011835157, 0.030800631, -0.012346553, 0.008043524, 0.0037697167, 0.0031158607, -0.005954107, -0.00079996896, 0.0054536695, -0.0029989704, -0.0017524434, -0.016423106, 0.030040843, -0.037667945, 0.045411937, 0.0033715586, -0.015181146, 0.024605438, 0.011426041, 0.0051212623, -0.011886297, -0.007590573, -0.019535314, 0.0017716207, 0.02809754, 0.0030647211, 0.012514583, -0.021098724, 0.018556358, 0.047165293, 0.005252764, 0.039771974, 0.029295666, -0.009891854, -0.010680864, 0.0037879809, 0.0071120528, 0.015780209, -0.027615367, -0.0027085706, -0.038573846, -0.0052089305, 0.005844522, 0.010717392, -0.01700756, -0.015298036, 0.012346553, 0.043717023, -0.026285738, -0.010834282, 0.02172701, -0.0068709664, -0.023305032, 0.022340685, -0.014618611, 0.022413742, -0.0053221676, -0.009380458, 0.014633222, 0.040327203, 0.017562788, -0.039655082, 0.0013533721, 0.018322576, -0.05742243, -0.025058389, 0.005566907, 0.011133814, 0.025788954, 0.008145803, 0.009526571, 0.03492102, -0.03012851, -0.02139095, -0.0035925547, 0.02071883, 0.014808558, 0.0020035752, -0.029047275, 0.012916394, 0.033343, 0.01769429, -0.006352265, 0.018366411, -0.014757418, 0.018118018, 0.037229605, -0.020046711, -0.0045368103, -0.0047596325, 0.015853265, -0.00036048828, -0.0218439, -0.0017588359, 0.004821731, 0.008832534, -0.018381022, -0.012646085, -0.005566907, 0.011900908, 0.024561604, 0.015750987, -0.010768532, -0.011776712, 0.011630598, -0.002524103, -0.01080506, 0.0072179846, -0.02115717, -0.0018510697, -0.013332816, -0.012383081, 0.03401512, 0.019418424, -0.020645775, -0.032203317, -0.020762665, -0.020075934, -0.010878117, 0.029310279, 0.02730853, 0.008467251, -0.022267628, -0.013047895, -0.036557484, 0.0026336878, -0.003762411, -0.0033624265, -0.017299784, 0.026388017, 0.033167664, 0.04108699, 0.037200384, 0.00072828226, 0.014026853, -0.02608118, -0.0072618187, -0.006103873, -0.015108089, -0.0056618806, -0.06002324, 0.013464318, 0.02518989, 0.03492102, 0.055902854, 0.027805313, -0.010366721, 0.010768532, 0.0037551054, 0.052016247, 0.019067753, 0.019184643, -0.01075392, -0.0020181865, -0.0057276315, -0.019228477, -0.019885987, 0.0126972245, 0.020996446, -0.017182894, 0.01277028, 0.04208056, -0.011564848, 0.02283747, -0.032963105, 0.032407876, 0.0015186626, -0.0137054045, -0.034804128, -0.027819924, -0.030742185, 0.042197447, -0.0057714656, -0.0027834536, 0.019579148, 0.025350614, -0.0076636295, 0.014092604, 0.0007912935, -0.0069695925, -0.02955867, -0.029500226, -0.013939185, -0.001370723, -0.04018109, 0.008942119, 0.0017268736, -0.0069038416, -0.01689067, -0.012675307, -0.047369853, -0.04748674, -0.007875494, 0.04275268, -0.009227039, -0.013771155, 0.03369367, -0.008043524, -0.00017487905, 0.0049714968, -0.041583773, 0.00035135623, 0.0004312618, 0.03135586, -0.027834537, 0.042226672, -0.03313844, -0.011294539, -0.009979521, 0.012141994, 0.047399074, 0.021551674, 0.0026592575, -0.024444714, 0.02529217, -0.022998193, 0.019199254, 0.016773779, -0.0073933206, -0.012025104, -0.019447647, 0.022793636, 0.0020218394, 0.005990635, 0.009000564, -0.029500226, 0.022559855, 0.025219113, 0.030391514, 0.0036838753, 0.007210679, 0.011382206, -0.020061322, -0.03763872, 0.007634407, -0.012047021, -0.014151049, 0.009446208, 0.021916958, -0.0133766495, 0.024532381, 0.009263568, -0.024985332, 0.03962586, -0.024473935, 0.001923213, -0.00046596365, -0.01927231, 0.023129696, -0.011630598, 0.020105155, 0.022632912, -0.0061696237, -0.002567937, -0.011462569, -0.003156042, -0.015546429, 0.0005862786, -0.008364972, -0.027790703, -0.0046756174, 0.010264441, -0.015210369, 0.0018994696, -0.023012806, -0.030508405, -0.016379273, 0.038632292, 0.011521014, -0.036440596, -0.022121515, -0.01915542, -0.0016611228, -0.014764723, -0.0099137705, 0.02922261, 0.044973597, -0.048655644, -0.027396196, 0.0068782717, -0.010249831, 0.042694233, 0.0114698745, 0.014764723, -0.0019834847, -0.036878932, -0.0035048868, 0.005187013, 0.0063887932, -0.023801817, -0.040911656, 0.0052710283, -0.02989473, 0.04424303, 0.0070937886, -0.00025820915, 0.008153108, 0.0032564946, -0.01971065, 0.003280238, 0.0032290984, -0.008533002, 0.03831084, -0.008978647, -0.020806499, 0.028345931, -0.025160667, 0.017153673, 0.0071120528, 0.013354733, -0.0024967068, -0.01500581, -0.00056207867, 0.0044929762, -0.0012912741, 0.0008894632, 0.009899159, -0.01779657, 0.018950863, 0.015604874, 0.08655737, -0.031297415, -0.032846212, -0.016773779, -0.0018556358, -0.004408961, -0.013369344, -0.022720579, -0.03144353, -0.0059394953, -0.017665068, 0.028681992, 0.00035911848, 0.018381022, 0.016525386, -0.012295413, 0.0028802536, -0.0012794024, 0.055435292, 0.013274371, 0.020572718, 0.055435292, 0.001198127, -0.0058847032, 0.01393188, 0.019418424, 0.026329571, -0.0014291683, 0.0019871376, 0.032875437, 0.0079777725, -0.044096917, -0.023480367, 0.01169635, -0.023904094, -0.00079494633, 0.014246022, 0.025935067, -0.027060136, 0.021347117, -0.02787837, -0.0012757495, 0.026519518, 0.00094060274, 0.01734362, 0.009855325, 0.0026062916, 0.015838655, 0.010870811, -0.017606623, -0.028521268, 0.0059175785, -0.0029076496, -0.017504344, 0.026358794, -0.05102268, 0.012324636, 0.018161852, 0.012711835, 0.0051431796, -0.032086425, 0.01924309, -0.02730853, 0.02114256, -0.0024875747, -0.060490802, -0.0053112092, 0.0037879809, 0.006958634, 0.03427812, -0.02888655, -0.010797755, 0.01416566, -0.0010775838, 0.021464007, -0.00043902406, -0.017153673, -0.0018026697, 0.038603067, 0.022384519, -0.013654265, -0.024108654, -0.026490295, -0.025701286, 0.009731129, 0.017445898, 0.049269322, 0.008204248, -0.04257734, 0.020338936, -0.027337752, 0.034862574, 0.04827575, 0.04792508, 0.024707716, -0.008518391, -0.0020693261, -0.010257136, 0.0032510154, -0.014238717, 0.028448211, 0.009007869, -0.028258264, 0.009175899, -0.019564537, -0.010980396, -0.036995824, 0.0012565722, 0.00929279, 0.016525386, 1.4204643e-05, 0.02159551, 0.008657198, -0.0061842347, -0.006418016, 0.02128867, -0.003311287, 0.024444714, -0.006732159, -0.022486798, 0.0153418705, 0.012850643, -0.0116598215, -0.011484486, -0.0035907282, -0.024269378, 0.036294483, 0.0029460043, 0.0053733075, 0.0138296, -0.010563973, -0.0010008744, 0.022881303, -0.011732878, 0.0317942, -0.0071120528, -0.020908777, -0.012047021, -0.017679678, 0.037200384, 0.0026903066, 0.011637905, -0.042957235, -0.018278742, -0.0068198266, 0.005636311, 0.008847145, 0.007692852, 0.011586765, 0.021098724, -0.023889484, -0.034336567, 0.021814678, -0.019681428, -0.0038610373, -0.03582692, -0.015049644, 0.011126509, -0.008657198, -0.016174715, 0.036265258, -0.004518546, -0.03194031, 0.016087048, 0.026504908, 0.0073275696, -0.026972469, -0.00012122817, 0.03635293, 0.022676745, -0.02159551, 0.010395943, -0.012551111, 0.0029642684, 0.0032510154, -0.021303283, 0.024663882, -0.011214176, -0.004854606, 0.019301534, -0.010154857, -0.03123897, 0.0013104514, -0.020017488, -0.017898848, 0.023772594, 0.01393188, -0.0046756174, -0.00979688, 0.024590826, 0.006973245, -0.019988265, -0.012200439, 0.005592477, 0.01935998, 0.0072910413, -0.02720625, -0.0058700917, 0.016013991, 0.016218549, 0.006253639, 0.006202499, 0.01668611, 0.022179961, -0.024956109, 0.01248536, 0.0035615056, -0.0031779588, -0.013975713, 0.00979688, -0.032057203, 0.0055048093, -0.032846212, -0.011024229, 0.021405563, -0.018877806, 0.012689918, 0.001030097, -0.0055632545, -0.00095338764, 0.011652516, 0.0076417127, 0.023568034, -0.0020766316, -0.0376095, 0.013172092, 0.01824952, -0.033167664, 0.017168283, 0.0070134266, -0.0075759618, -0.03348911, -0.007933939, -0.0012419609, 0.0041532633, -0.0012739232, -0.0037733696, -0.014903531, -0.009526571, 0.0029058233, -0.0074298484, 0.034804128, 0.017723514, 0.01689067, -0.008810617, 0.064523526, -0.013033284, -0.0018254999, -0.010140246, -0.017898848, 0.025788954, 0.030362291, -0.030274624, -0.015531817, -0.024123264, -0.016116269, -0.011586765, -0.008598753, -0.020850332, 0.03804784, -0.01656922, -0.008240776, 0.0062353746, 0.0023195448, -0.009351235, 0.017314397, 0.019286923, -0.025613617, -0.011586765, 0.013902657, -0.010366721, -0.0077293804, -0.03591459, -0.028506655, 0.017109837, 0.02809754, -0.0060088993, 0.010162163, 0.011236094, -0.011228788, 0.014077992, -0.03214487, -0.017855015, 0.0024784426, -0.029529447, -0.0022446618, 0.007020732, -0.0051212623, -0.020017488, -0.010498223, 0.026592575, 0.024327822, 0.017781958, -0.007071872, 0.026110401, -0.014377524, -0.01298945, -0.014830475, -0.011710961, 0.016773779, -0.041671444, -0.015429538, 0.01169635, -0.009869937, -0.0004958712, 0.0043212934, -0.010783143, 0.05791921, -0.010096412, -0.019096976, 0.031005189, -0.00979688, 0.004664659, 0.0046938816, 0.033343, 0.013698098, 0.013603125, -0.012054327, -0.040531762, -0.02206307, -0.016393885, 0.040297978, 0.016613053, 0.0012994929, 0.0068782717, -0.017036783, -0.0026446462, 0.007090136, 0.028287487, -0.0073421807, 0.021127947, 0.022384519, 0.014151049, 0.013836906, -0.0007237162, -0.014399441, 0.0036984866, -0.02283747, 0.02575973, -0.007006121, 0.013427789, -0.0074809883, -0.02955867, 0.0166715, -0.024488548, 0.047662076, -0.036469817, 0.01824952, -0.015444149, -0.002334156, 0.0038793015, 0.006874619, -0.000499524, -0.05695487, -0.040823985, -0.011849768, 0.0076709352, 0.01914081, 0.03237865, -0.024780773, -0.059701793, 0.0113602895, 0.025803564, -0.0055413377, -0.024488548, 0.01824952, -0.0029496572, -0.03381056, -0.015911711, -0.014267939, 0.0055596014, 0.0061732763, 0.013069812, -0.018307965, -0.03950897, -0.005574213, -0.022048458, -0.0035395885, 0.010016049, -0.021873124, 0.025569784, -0.028024483, 0.055259954, -0.014034159, 0.032963105, -0.01656922, -0.029529447, 0.022004625, -0.0015688889], "content_hash": "43ebc430ddce5bcd25705c23092396a5"}, {"content": "1. Look at **Stage 5 (Output)** to see the top-5 predictions for the immediate next token. 2. Note the top prediction and its probability. Are the alternatives close in probability? If so, the model was uncertain, which explains why beams diverge early. ### Step 5: Ablate a Head and Re-Generate\n\n1. Go to the **Ablation** tab in the Investigation Panel.\n2. From the head categories in Stage 3, pick a **Previous-Token** head (e.g., L0-H3).\n3. Add it and click **\"Run Ablation Experiment.\"**\n4. Look at the **Full Generation Comparison**:\n   - Did the ablated generation diverge from the original?\n   - Did the model take a completely different path, or just change a word or two?\n   - Did the ablated generation match one of the other beams you saw earlier?\n\n### Step 6: Try a Stronger Ablation\n\n1. **Clear** the selected heads.\n2. Add **two or three** Previous-Token heads from different layers.\n3. Run the ablation again.\n4. Compare: does ablating multiple heads cause a bigger divergence than ablating one?\n\n### Step 7: Experiment with Different Beam Settings\n\n1. Change the prompt to: `The scientist discovered that the`\n2. Try with **1 beam** (greedy decoding): note the single output.\n3. Try with **3 beams**: see the alternatives.\n4. Try with **5 beams**: do you get even more diverse options?\n\nNotice how increasing beams reveals the model's uncertainty -- places where multiple continuations are roughly equally likely.\n\n## What You Should Learn", "source_file": "experiment_beam_search.md", "chunk_index": 1, "embedding": [0.039442945, 0.013578846, 0.05983058, -0.0073781237, -0.03699519, 0.008063651, -0.0016712128, 0.02311425, 0.016235746, 0.059706643, 0.027622456, -0.02976037, -0.06562464, -0.02407476, 0.015406917, -0.023362122, 0.014345706, -0.016065333, -0.02765344, 0.08192235, -0.02895478, -0.039195072, 0.022153737, 0.031170152, 0.026476039, -0.027312614, 0.014694279, 0.047870662, 0.023005804, -0.012385953, -0.0018832613, -0.026150705, -0.026538007, -0.011688807, -0.056856092, 0.003336616, 0.006258818, 0.03093777, -0.018962359, 0.009783276, 0.02523667, -0.034175623, -0.042045623, 0.03349397, 0.0131915435, 0.011944427, 0.012819733, -0.05165074, -0.010255786, 0.03742897, 0.014694279, 0.05484212, -0.026816865, -0.003993095, -0.035291057, -0.005871515, -0.02079043, -0.041456923, 0.014903423, -0.05165074, 0.005770816, -0.00087433675, 0.03165041, 0.02224669, 0.0016896097, -0.0062781833, -0.032440506, 0.042045623, 0.0071070115, 0.039257042, -0.0011851474, -0.013695037, 0.010782518, 0.012432429, -0.025794385, -0.0067042164, 0.053757668, -0.0030674406, -0.016808955, -0.008435462, 0.0458257, 0.0068397727, 0.050318416, -0.034392513, 0.022664977, -0.042138577, -0.0005102718, -0.035353027, -0.060667157, -0.06488101, -0.020589033, 0.00031008455, -0.05143385, 0.012354969, 0.013911927, -0.0059877057, 0.031603932, -0.050287433, 0.025159208, 0.0069249794, 0.01579422, 0.014384436, -0.016576571, -0.0010205436, 0.06299098, 0.0064679617, -0.044524364, 0.014345706, -0.03149549, 0.0022405484, -0.04762279, -0.017382164, -0.022819899, 0.008714319, -0.012889447, -0.03860637, -0.051371884, 0.037459955, 0.032440506, -0.01473301, -0.054408338, -0.014384436, -0.0024535651, 0.020914366, 0.01736667, -0.01528298, 0.07826621, -0.03715011, -0.0066151368, -0.0010186072, 0.0023005803, -0.004120905, 0.03575582, -0.024245173, -0.028939286, 0.022850882, -0.018946867, 0.03931901, -0.05332389, 0.041178066, -0.009233305, -0.02057354, -0.00032364015, -0.011712045, -0.028521, 0.029589957, -0.0043377946, 0.044028614, -0.038885232, 0.019644013, 0.032626413, -0.013036622, -0.013625323, -0.026212674, 0.0061968495, -0.033648893, -0.041456923, 0.023548028, -0.030178657, -0.04539192, -9.234758e-05, 0.043966647, -0.017738482, -0.0008351223, -0.03115466, 0.053602748, -0.02560848, 0.03553893, -0.036499444, -0.038854245, -0.021131257, -0.011146583, -0.0059257373, -0.0040860474, 0.0010767026, 0.003144901, -0.0938203, -0.074238256, -0.031464502, -0.057723653, -0.0010466865, 0.033060193, -0.01736667, -0.0041828733, -0.012571858, -0.025701433, 0.003152647, -0.0028040744, -0.017645529, -0.011804998, -0.013222528, 0.021905864, 0.062278338, -0.008458699, -0.030070212, -0.023176217, 0.06413739, 0.028613953, 0.010193817, -0.010178326, 0.037181098, -0.0017331813, 0.03801767, -0.006890122, 0.016142793, 0.008234064, 0.01846661, 0.014252754, -0.036220584, -0.009883975, 0.028722398, -0.040651333, 0.031975742, 0.01852858, -0.044152554, -0.004759955, -0.0022347388, 0.050814167, 0.021069288, 0.049853653, 0.035910744, -0.010681819, 0.027498519, 0.071139835, -0.0033869655, 0.005867642, 0.05326192, -0.024647968, 0.04567078, 0.022974819, 0.03532204, 0.009155845, -0.00691336, -0.01371053, -0.0005611054, -0.032285586, -0.0132302735, -0.028087221, 0.02413673, -0.06091503, -0.015042852, 0.012982399, 0.037367, -0.008636859, -0.026026769, 0.04610456, -0.03882326, -0.027529504, 0.0077499347, 0.034299564, 0.039659835, -0.015856188, -0.022494564, -0.008249556, 0.0066809785, -0.008396731, -0.043439914, -0.0059605944, -0.040434442, -0.015515362, -0.0114486795, 0.020914366, -0.05267322, 0.01640616, -0.04297515, 0.014322468, -0.01579422, -0.011758522, -0.040372476, -0.06500495, -0.024105744, -0.036220584, -0.0042758263, -0.008319271, -0.04436944, -0.029698402, -0.018621532, -0.00081382063, -0.02391984, -0.0009726149, -0.012029634, -0.017041337, 0.035414994, -0.010867725, -0.0012074173, 0.031170152, -0.009907213, -0.01360983, -0.0052092266, -0.029419543, 0.03699519, -0.03876129, 0.038637355, -0.03318413, -0.008288286, 0.001566641, 0.011595855, 0.013695037, -0.06735975, -0.0037064906, 0.018621532, -0.03237854, 0.050225466, 0.002521343, -0.016096316, 0.0012529255, 0.00426808, 0.02378041, 0.00094647193, 0.00038270387, 0.02311425, 0.039349996, -0.02771541, -0.0434709, -0.022757929, -0.043223023, 0.04691015, -0.051774677, 0.024322635, -0.05041137, -0.022742437, -0.026909819, -0.01331548, 0.019148264, 0.0019374837, -0.01276551, -0.01221554, -0.030287102, 0.008466446, -0.028335094, 0.021348147, 0.009605116, 0.041116096, 0.0019248964, -0.058405306, -0.01618927, -0.0048374156, 0.03922606, 0.037026174, 0.02991529, -0.054191448, 0.014802724, 0.02641407, -0.021859387, -0.011386711, 0.024570508, 0.0028795984, -0.007738316, 0.030596944, 0.037274048, -0.03420661, -0.022262182, -0.008358001, -0.012912685, 0.044028614, 0.0479946, 0.024260666, -0.03392775, 0.035291057, 0.00022608817, -0.02472543, 0.06128684, 0.033741847, 0.03268838, 0.019349663, -0.020635508, -0.011611347, -0.00015334782, 0.004217731, -0.0055384343, -0.027823854, 0.0025891212, -0.021673482, 0.028195664, -0.097662345, -0.0059489757, 0.0006322723, 0.023904346, 0.006030309, 0.072627075, 0.0058250385, -0.038916215, -0.053664718, -0.051247943, 0.019752458, 0.035043184, -0.02633661, 0.0070566623, -0.03603468, 0.0013090845, 0.02429165, -0.0123394765, 7.606875e-05, 0.00601869, -0.007753808, 0.03990771, -0.035848774, 0.057227906, 0.030287102, 0.053107, 0.019783441, -0.024709938, -0.060976997, -0.029899798, 0.0096670855, 0.026383087, 0.045113064, 0.011200805, 0.051991567, 0.023656473, 0.041673813, 0.007633744, 0.05564771, -0.031464502, -0.0024264539, 0.038110625, 0.040744286, 0.030333579, 0.02429165, 0.04427649, -0.021162242, -0.0735566, -0.030116688, -0.046507355, -0.029435035, -0.027002772, -0.020620016, -0.011386711, 0.06265015, -0.044121567, 0.0098917205, 0.056949046, 0.04356385, -0.013656307, -0.009945943, -0.008350255, 0.029450526, 0.048397396, -0.045918655, -0.029450526, 0.022943836, 0.00020406031, -0.027095724, -0.032440506, -0.05121696, 0.019008836, -0.06618235, -0.04173578, 0.0030868056, -0.03879228, -0.029589957, -0.0361896, -0.025081748, 0.054377355, -0.0052285916, -0.04325401, 0.09059794, -0.0084044775, 0.08458699, 0.00023322908, 0.022432595, -0.028846335, -0.017939879, -0.0017709434, -0.01846661, 0.041054126, 0.01969049, -0.030395547, 0.018869407, 0.024833875, 0.020976335, -0.028242141, -0.02299031, -0.050752196, 0.007629871, -0.0006588994, 0.043099087, 0.0026220419, 0.030767357, -0.031820823, -0.013493639, 0.00036842207, 0.039133105, 0.04223153, -0.0040783016, 0.08626015, 0.004798685, -0.00798619, 0.003189441, -0.01721175, 0.008853748, -0.04669326, 0.026491532, -0.020960843, 0.062216368, -0.028397063, -0.042572357, 0.0007983285, 0.009155845, 0.006804915, 0.057537746, -0.020449603, -0.01618927, 0.018141277, 0.03312216, -0.017645529, 0.03495023, 0.035848774, -0.010852233, 0.066864006, -0.019876394, -0.037119128, -0.00060467696, 0.005410624, -0.027483027, 0.014105579, -0.0018309754, 0.036437474, 0.022556532, 0.028505508, -0.020186238, -0.011975411, 0.007839015, 0.030906787, 0.014887931, -0.032006726, -0.036685348, -0.036902238, -0.021363638, 0.010364231, 0.035198104, -0.04185972, 0.0029241384, 0.024555016, -0.040961176, -0.047591805, -0.01626673, 0.021177733, -0.024586, -0.01648362, -0.0015869744, 0.029047731, 0.0017854673, 0.0009614799, 0.0072967904, -0.0066654864, 0.03349397, 0.0062084687, 0.007304536, 0.0029919164, 0.012657065, -0.007134123, 0.0026530263, -0.03829653, -0.001510482, 0.0044849697, -0.025159208, -0.025469052, -0.0028641063, -0.028288618, 0.021503069, -0.012308492, -0.005023321, -0.013307734, -0.016746985, 0.013013384, -0.009636101, 0.0050620516, 0.004655383, 0.005151131, -0.024369111, -0.029063225, 0.004798685, 0.023300154, 0.0066345017, 0.008086888, 0.030875802, 0.021719957, 0.014469643, -0.0030345197, -0.014043611, -0.039659835, -0.04093019, 0.01280424, -0.029202653, -0.029698402, 0.0014630374, 0.030720882, -0.013323226, 0.014105579, 0.035879757, -0.012943669, 0.015205519, 0.024926826, -0.016808955, -0.008644605, 0.015143551, 0.0063091675, 0.016592065, -0.01105363, 0.031108184, -0.008342508, -0.021998815, 0.015368187, -0.03721208, 0.011378964, -0.016623048, 0.0022967074, 0.031696886, -0.0037181096, -0.01838915, 0.009791022, -0.053137984, -0.024616985, 0.0024845493, -0.014903423, -0.0132690035, 0.014275992, 0.04777771, -0.013842212, 0.0077228234, 0.0058289114, -0.009651593, 0.015406917, 0.017599052, 0.0029125193, -0.014562597, -0.016111808, 0.013772498, 0.022479072, -0.038885232, 0.01502736, 0.013400687, -0.0023373743, -0.013408433, 0.016468128, 0.0052440837, 0.029791353, -0.018698994, -0.011766268, 0.0016876733, 0.016142793, 0.012819733, -0.008838257, -0.036964208, 0.0054609734, 0.012347222, -0.00119483, -0.025840862, -0.048552316, -0.03259543, -0.033308066, 0.06661613, -0.009031908, 0.020480588, -0.013950657, -0.0029861068, -0.028644936, 0.0025310258, -0.018079309, -0.011518394, 0.025407083, -0.031619426, -0.03296724, -0.0045663035, 0.03457842, 0.01618927, 0.03938098, 0.00700244, 0.0025174702, -0.021053797, 0.024353618, -0.021921355, 0.016592065, -0.040372476, 0.0025310258, 0.007467204, -0.033091176, -0.00995369, 0.012657065, 0.0011715918, 0.01567803, -0.06990046, -0.040713303, -0.0004831606, -0.007924221, -0.0001341037, -0.031092692, 0.0030364562, -0.0059257373, -0.023842378, -0.008846003, 0.009729054, 0.0126725575, -0.003489601, -0.01105363, -0.007242568, 0.02976037, -0.0057165935, -0.06723582, 0.00086465414, -0.00849743, 0.008388985, -0.018079309, -0.016111808, -0.01919474, 0.0025484543, 0.061100937, 0.021859387, -0.017149782, -0.004368779, -0.015631553, -0.021146748, 0.020852398, -0.041023143, 0.012695795, -0.0049226223, -0.008621367, -0.009016416, -0.04142594, -0.018373659, 0.028180173, -0.035848774, -0.0034624897, 0.018652517, -0.031820823, -0.012440176, -0.000913067, -0.021069288, 0.01174303, 0.01127052, 0.016313206, 0.01272678, -0.022432595, -0.0025348987, 0.030240625, -0.01882293, -0.0004165929, -0.008985432, -0.01882293, 0.03959787, -0.020743953, -0.0009953689, 0.022091769, 0.024152221, -0.0032029964, 0.009055146, 0.016901907, -0.028753381, -0.015763236, -0.017242733, 0.0155308535, -0.0042022387, -0.01589492, 0.05326192, 0.006901741, 0.010023404, 0.000856424, 0.01557733, 0.0117043, 0.024028284, 0.0060845315, 0.004512081, 0.022277674, -0.012385953, 0.0085903825, -0.0132690035, 0.021952339, 0.0110149, 0.03420661, -0.010712803, 0.019814426, 0.017753974, 0.010581121, 0.033958737, -0.009395973, 0.037088145, -0.0153062185, -0.051371884, 0.0029512495, -0.02370295, -0.00477932, -0.03820358, 0.021503069, -0.015925903, 0.00585215, -0.017521592, -0.016437143, -0.022262182, -0.028164681, -0.0026685183, 0.005917991, 0.020232713, -0.020883383, -0.010720549, 0.02079043, -0.0059218644, -0.11959919, -0.0068010422, -0.0008007491, 0.008884733, -0.01998484, -0.023548028, 0.03473334, -0.0018542135, -0.017599052, 0.027157692, -0.013849959, -0.032564443, -0.0010892899, -0.00268982, 0.013973895, -0.033215113, 0.0033540449, -0.023021296, -0.018791946, -0.027839346, -0.030070212, 0.010325501, -0.0064369775, -0.009016416, -0.04675523, 0.020108776, -0.022076277, -0.011084614, 0.003580617, -0.004120905, -0.005522942, 0.042789247, -0.017970864, 0.034454484, 0.010488167, -0.015941394, -0.011750775, -0.002641407, 0.0059528486, -0.0017999911, 0.018327182, -0.029311098, 0.025221176, -0.031247614, 0.019210232, -0.01422177, -0.029047731, 0.018420136, 0.034299564, -0.019736966, -0.016375175, -0.015228758, -0.0107437875, 0.0045624305, 0.034454484, 0.012161317, -0.00900867, 0.005623641, -0.005038813, 0.029218145, 0.023377614, 0.046507355, 0.03377283, -0.010217056, 0.00060080393, -0.024338126, 0.029961767, 0.014252754, -0.035848774, -0.03575582, -0.018063815, 0.0017264035, -0.0076531093, 0.01473301, -0.00033840607, 0.01455485, 0.039721806, 0.029342081, -0.006061293, -0.018187754, 0.0277309, 0.009798768, -0.02480289, 0.027235154, 0.0023083263, 0.008141111, -0.006502819, -0.017087812, 0.020263698, 0.033091176, 0.027390074, -0.020511571, -0.0014756247, 0.018729977, -0.063207865, -0.02763795, -0.0009106464, -0.0017080066, 0.021750942, 0.0053254175, 0.0155308535, 0.035074167, -0.017939879, -0.017413147, 0.00060467696, 0.022711454, 0.0018958487, 0.007137996, -0.01961303, -0.0039214436, 0.04040346, 0.010619851, -0.005170496, 0.025716925, -0.033741847, 0.02830411, 0.026832359, 0.011045884, -0.009883975, -0.013346464, 0.0058753877, 0.004167381, -0.027188677, 0.010952932, -0.00537964, 0.002869916, -0.009504418, 0.0023238186, 0.002825376, 0.008807273, 0.0058908802, 0.016716002, -0.012323984, 0.022386119, 0.0054648467, -0.008303778, 0.006254945, 0.004407509, -0.023749426, 0.019783441, -0.021611512, -0.0052440837, 0.02735909, -0.002957059, -0.009798768, -0.011866966, -0.041332986, -0.021936847, 0.0033327432, 0.027018264, 0.04254137, 0.014942153, -0.013137321, 0.0048296694, -0.026197182, 0.019458108, 0.02269596, -0.0063091675, -0.008079142, 0.040062632, 0.042510387, 0.026909819, 0.028877318, -0.018234229, 0.012866208, -0.0060651666, -0.005476466, -0.02378041, -0.0029144557, -0.0013962276, -0.050752196, -0.002318009, 0.017165273, 0.031619426, 0.044338457, 0.021998815, -0.0107050575, 0.016654033, -0.0050349403, 0.04399763, 0.031216629, 0.026150705, -0.021673482, -0.0021863258, 0.018714486, -0.01852858, 0.006495073, 0.025995784, 0.029125193, -0.010736042, -0.0017118796, 0.03764586, -0.009380481, 0.029589957, -0.019287694, -0.0021921354, 0.0076221246, 0.026739405, -0.033215113, -0.014717517, -0.030968755, 0.015104821, -0.0032591554, 0.00029289795, 0.009426957, 0.011518394, -0.017010352, 0.020465095, -0.0012500207, 0.011549378, -0.03129409, -0.02371844, 0.0065996447, -0.005499704, -0.03172787, -0.0038013798, 0.012904939, -0.004368779, -0.0050349403, -0.01502736, -0.034485467, -0.038110625, -0.0031507106, 0.026026769, -0.0012083857, -0.00058143877, 0.036809284, 0.0027208042, -0.0046979864, -0.003996968, -0.01801734, -0.0103719765, -0.0005833753, 0.010333247, -0.029435035, 0.029713893, -0.049048062, -0.0073626316, -0.026646452, 0.013431671, 0.05608149, -0.006510565, -0.02721966, -0.01626673, -0.0013265131, -0.015956888, 0.00040836268, 0.014438659, -0.01360983, -0.024678953, -0.013307734, 0.008125619, 0.023749426, 0.003257219, 0.008195333, -0.020356651, 0.020294681, 0.021022812, 0.03420661, 0.0073316474, 0.013346464, 0.019427123, -0.01506609, -0.026956296, 0.0048722727, -0.020248206, -0.015050598, 0.009605116, 0.036592398, -0.027839346, 0.011386711, 0.029373066, -0.004643764, 0.029357575, -0.022664977, -0.0061658653, 0.009612863, -0.02924913, 0.01247116, 0.00011092603, 0.021936847, 0.0005615895, -0.028582968, -0.005399005, 0.0025736291, -0.0034063307, -0.014779486, 0.007292917, -0.0007547569, -0.022200214, -0.002141786, 0.0096670855, -0.008613621, 0.0011105916, -0.009326259, 0.0008786939, -0.021565037, 0.03275035, 0.033215113, -0.06897093, -0.0050504324, -0.003433442, -0.0003388902, -0.020542556, -0.002205691, 0.012401445, 0.046073575, -0.050225466, -0.030953264, -0.0058444035, -0.0021379131, 0.033834796, 0.010852233, 0.015190028, -0.005755324, -0.043935664, -0.0015337203, 0.0081101265, 0.0054222434, -0.022076277, -0.033277083, -0.0022076275, -0.012966908, 0.030008243, -0.015430155, -0.026104229, 0.004488843, 0.002577502, -0.0045546843, 0.02079043, 0.020743953, -0.001826134, 0.028567476, -0.0033579178, -0.015832951, 0.030085703, -0.034454484, 0.0074246004, -0.01567803, -0.012362715, 0.0081101265, 0.028521, -0.0094811795, 0.004411382, -0.0019694362, 0.00018324278, 0.014245008, -0.011177567, 0.016468128, 0.0010941313, 0.09729054, -0.037831765, -0.050752196, -0.02181291, -0.002957059, -0.009876229, -0.00024266959, -0.023315646, -0.0389472, -0.004752209, -0.005495831, 0.024864858, 0.0024671208, 0.02071297, 0.021611512, -0.00601869, 0.01911728, -0.0073122825, 0.039690822, -0.0059102452, 0.021394623, 0.04734393, -0.013330973, -0.0041828733, 0.007656982, 0.017196257, 0.013098591, 0.0036387125, -0.004639891, 0.048397396, -0.029682908, -0.032874286, -0.032068696, 0.018280705, -0.017134288, -0.0071689803, 0.009395973, 0.018962359, -0.035198104, 0.019055312, -0.028397063, -0.011286012, 0.01911728, 0.004171254, 0.014539358, 0.00554618, -0.0005887007, 0.005879261, 0.00047517248, -0.024183204, -0.0040356982, -0.0042293495, -0.0056933556, -0.011797252, 0.03377283, -0.043130074, 8.744578e-06, 0.025205685, 0.012657065, 0.004748336, -0.026088737, 0.0016344191, -0.012192301, 0.030473007, 0.0060031977, -0.064819045, 0.00081139995, 0.00060612935, -0.0092952745, 0.048180506, -0.03143352, -0.013989388, 0.011510648, -0.00977553, 0.0111155985, -0.017428638, -0.013090844, -0.007064408, 0.014880185, 0.014237261, -0.019628521, -0.03857539, -0.036437474, -0.031820823, -0.011185314, 0.012502144, 0.05887007, 0.022881867, -0.02625915, 0.011619093, -0.035074167, 0.026507024, 0.045639798, 0.043656804, 0.012393699, -0.035136137, -0.003185568, -0.014415421, -0.00563526, 0.0006990821, 0.014887931, 0.0041790004, -0.03786275, 0.00652993, -0.027761886, -0.028288618, -0.0344235, -0.006398247, 0.0057282127, 0.01174303, 0.010193817, 0.012091602, 0.02159602, 0.0053099254, -0.005143385, 0.029094208, 0.015809713, 0.03693322, -0.013888689, -0.034888264, 0.011975411, 0.02611972, -0.010875471, -0.036313538, -0.002254104, -0.025143716, -0.00798619, 0.017784959, -0.0015908475, 1.7368124e-05, -0.0024477555, 0.0052285916, 0.013361957, -0.0057127206, 0.02342409, 0.0011928935, -0.011812745, 0.009419211, -0.021115765, 0.043966647, 0.004883892, 0.008644605, -0.030906787, -0.039411962, 0.01903982, 0.010069881, -0.0024787397, -0.0025407083, 0.0036561412, 0.01948909, -0.019055312, -0.01567803, 0.01917925, -0.015507616, 0.0020488333, -0.043501884, 0.0006971456, 0.00082059845, 0.0027227406, 0.00055142277, 0.034330547, 0.013478148, -0.040961176, 0.016452635, 0.020403126, 0.0043300483, -0.03705716, 0.0069482173, 0.03823456, 0.002893154, -0.02100732, -0.013423925, -0.0039020786, -0.0054183705, -0.012626081, -0.028366078, 0.014051356, 0.015468885, 0.0069288523, 0.0464144, -0.014918915, -0.0073471395, 0.029775862, 0.01787791, -0.015120313, 0.008303778, 0.005766943, 0.00031758854, -0.007746062, -0.004368779, 0.011262774, -0.01579422, -0.012486652, 0.0011890205, 0.021657988, 0.015368187, -0.024973303, 0.010193817, -0.006777804, -0.0022115007, 0.010488167, -0.0077654268, 0.012323984, 0.0092952745, -0.026739405, 0.043966647, -0.001758356, -0.008017174, -0.0059722136, 0.016839938, -0.018420136, 0.0025794385, -0.01897785, -0.014337961, 0.044555347, -0.024539525, 0.003013218, -0.010674073, 0.00078622525, -0.012633827, 0.017304702, 0.014136563, 0.029125193, 0.012192301, -0.034454484, 0.0072115837, -0.01481047, -0.017041337, 0.017397655, -0.00027619553, -0.0026685183, -0.03640649, -0.022928342, 0.009651593, 0.0002885408, -0.0034295688, -0.02472543, -0.013873197, 0.00019232019, 0.015197773, 0.026088737, 0.005739832, 0.0059373565, 0.019860903, -0.007835141, 0.058901053, -0.010859978, 0.001382672, -0.012966908, -0.013633069, 0.015561839, 0.023145232, -0.025283147, -0.0039659836, -0.028970271, -0.028195664, -0.013640814, 0.0067274547, -0.043315977, 0.015035106, -0.0148259625, -0.0042874455, 0.005143385, -0.0051627504, 0.007873872, 0.007924221, 0.016173776, -0.021719957, -0.020743953, 0.014872438, -0.0049226223, 0.013284496, -0.031170152, -0.031820823, 0.018497596, 0.04458633, -0.0041635083, 0.011247282, 0.026305625, -0.02210726, 0.0066499943, -0.028644936, 0.013346464, -0.006464089, -0.00959737, 0.00930302, 0.005898626, 0.0107050575, -0.009326259, -0.0007315187, 0.022959327, 0.029682908, 0.03420661, -0.0008549716, -0.010402961, -0.004283572, 0.011433187, -0.025918324, 0.00849743, 0.016716002, -0.029698402, -0.017815942, 0.006014817, -0.0010912265, 0.018946867, 0.013385195, 0.0013478148, 0.034640387, -0.00029144558, -0.01699486, 0.027668932, -0.013540116, -0.01502736, 0.0048567806, 0.01910179, 0.018218737, 0.014214023, -0.023609996, -0.028242141, -0.008063651, -0.026661946, 0.019210232, 0.011642331, -0.0050852895, -3.7066116e-07, -0.013811228, -0.010170579, 0.0077809193, 0.01831169, -0.015104821, 0.007854506, 0.019504584, 0.016638542, -0.0023799776, 0.0022521676, -0.02210726, 0.010000166, -0.032068696, 0.012138079, -0.020232713, 0.0018454993, -0.0038343007, -0.0119676655, -0.015569584, -0.030953264, 0.027839346, -0.050721213, 0.0062084687, -0.01838915, 0.026677437, 0.0062975483, 0.012099349, -0.0018774518, -0.038885232, -0.012269762, -0.008505176, -0.0019045629, 0.026088737, 0.005646879, -0.040682316, -0.04783968, -0.007176726, 0.03873031, 0.0053137983, -0.015112567, 0.024260666, -0.008458699, -0.06271212, -0.02751401, 0.007544664, -0.0010912265, 0.027111216, 0.026894327, -0.016204761, -0.009605116, -0.007436219, -0.030488499, -0.007970697, 0.007494315, -0.013973895, 0.030287102, -0.014198531, 0.03597271, -0.012262016, 0.031479996, -0.018482104, -0.020015825, 0.008187587, 0.021270685], "content_hash": "623c172e26f32a3a32e5e31420b6b35a"}, {"content": "**1 beam** (greedy decoding): note the single output. 3. Try with **3 beams**: see the alternatives. 4. Try with **5 beams**: do you get even more diverse options? Notice how increasing beams reveals the model's uncertainty -- places where multiple continuations are roughly equally likely. ## What You Should Learn\n\n- **Beam search reveals model uncertainty**: When the model isn't sure, multiple beams show the different paths it's considering.\n- **Ablation can redirect generation**: Removing important heads can push the model from one beam to another, showing that different attention heads support different generation paths.\n- **More beams = more alternatives**: But beyond 3-5 beams, the additional paths are often low-probability and less interesting.\n- **Generation is a chain**: Each token depends on the previous ones, so a small change early (from ablation or beam selection) can cascade into a very different output.\n\n## What's Next?\n\nYou've now completed the core experiments. Try combining techniques:\n- Run attribution to find which input tokens matter, then ablate the heads that seem to process those tokens\n- Compare how GPT-2 and Qwen2.5-0.5B handle the same prompt with the same beam settings", "source_file": "experiment_beam_search.md", "chunk_index": 2, "embedding": [0.03050498, 0.020913439, 0.03920151, -0.014603606, -0.015319616, -0.021167025, -0.021017857, 0.03693415, 0.017124556, 0.07052692, 0.033652436, -0.011150354, -0.046152767, -0.006276269, 0.0024482298, -0.032280087, -0.013469925, -0.024254816, 0.0078015183, 0.073629625, -0.029162463, -0.019973677, 0.008144606, 0.035323124, 0.031922083, 0.0012539485, -0.005772825, 0.028088449, 0.012082657, -0.01525249, 0.012850875, -0.04120037, -0.022822797, -0.0107848905, -0.07094459, 0.018332822, -0.03230992, 0.01743781, -0.012090116, -0.014163559, 0.03320493, -0.02280788, -0.03753082, 0.036904313, 0.012619664, 0.004195367, -0.015095863, -0.032011583, -0.02009301, 0.036725312, 0.025791252, 0.08144606, -0.022613961, 0.01126223, -0.029818805, -0.008495152, -0.051105164, -0.03687448, 0.014506647, -0.033712104, -0.0063657705, 2.8653774e-05, 0.048628967, 0.042960558, 0.002865902, -0.02750669, -0.042274382, 0.025254246, -0.006376958, 0.036784977, -0.0033991795, 0.028655289, -0.023225551, 0.022032203, -0.01254508, 0.011142895, 0.047167115, 0.0028826834, -0.02793928, -0.03720265, 0.03648664, -0.0035035976, 0.034457948, -0.04770412, -0.0008045782, -0.03750099, 0.013335673, -0.0306989, -0.05698241, -0.040693197, -0.028148117, -0.0007104155, -0.055818893, 0.032906596, 0.011993156, 0.010039047, 0.016050542, -0.0412302, 0.009300662, 0.017497478, 0.019198, -0.002823016, -0.043169394, -0.005802659, 0.058921598, -0.012992586, -0.011016102, 0.014297811, -0.042005878, -0.044273242, -0.050389156, -0.011620235, -0.007689642, 0.014051682, -0.0024314483, -0.008830781, -0.01254508, 0.025000658, 0.046838943, -0.022121705, -0.036755145, -0.0061643925, -0.018899662, -0.003701246, 0.00032374248, -0.016766552, 0.051135, -0.04752512, -0.008122231, 0.0150809465, 0.014036766, 0.02304655, 0.039171677, -0.017885316, -0.04737595, 0.01952617, -0.03278726, 0.025269162, -0.057131577, 0.03824683, 0.016766552, -0.051612336, -0.013499759, -0.012052824, -0.044183742, 0.023315053, -0.023300136, 0.0205256, -0.030818233, 0.0010050235, 0.035382792, -0.024254816, -0.048479795, -0.028192867, -0.008681613, -0.042960558, -0.039947353, 0.05558022, -0.022628877, -0.04555609, 0.014141184, 0.050955996, -0.035860132, -0.009203703, -0.023195717, 0.038843505, -0.0358303, 0.043229062, -0.02974422, -0.041946214, -0.0028435267, -0.006376958, 0.00019496803, -0.006742421, 0.028565789, -0.0017331528, -0.09540824, -0.06300882, -0.042035714, -0.068140216, -0.002246852, 0.008927741, -0.016811302, 0.0073726582, -0.015513535, -0.025925504, 0.0045682886, -0.013335673, 0.0019783487, -0.010934059, -0.011515817, 0.036426973, 0.067603216, 0.011232396, -0.016423464, -0.041826878, 0.056863073, 0.049434476, 0.002142434, -0.024508402, 0.016706884, 0.0021256527, 0.043109726, -0.008607029, 0.021301277, -0.009427456, 0.006168122, -0.00094722066, -0.07374896, -0.016826218, 0.019496337, -0.0359198, 0.040693197, 0.017348308, -0.032906596, -0.0023643223, 0.027536524, 0.036218137, -0.004836792, 0.05596806, 0.02380731, -0.012574914, 0.03502479, 0.0410512, -0.0151033215, 0.035860132, 0.022420041, -0.020823937, 0.03663581, -0.009091826, 0.04018602, 0.011068311, -0.0014450708, -0.00026244353, -0.015162989, -0.021733865, -0.002228206, -0.021569781, 0.011336814, -0.08395209, -0.008778572, -0.0104492605, 0.031862415, -0.019556005, -0.030340895, 0.05757908, -0.04722678, -0.038306497, -0.015692538, 0.022464791, 0.017944984, -0.0057952004, -0.044303074, 0.01687097, -0.0032705218, -0.012507788, -0.022017287, 0.004911376, -0.023881894, -0.02617909, -0.016736718, 0.037888825, -0.028476287, -0.0031306762, -0.04689861, 0.027491774, -0.041021366, -0.028968543, -0.035382792, -0.06068179, 0.0016539069, -0.035084456, 0.016274296, -0.0013192098, -0.03559163, -0.033503268, -0.010091256, 0.005205984, -0.0283122, -0.011187646, -0.03359277, -0.014439521, 0.051552672, 0.0029237047, -0.015453868, 0.008465318, -0.016766552, 0.0022785505, -0.004855438, -0.022882463, 0.033473436, -0.018854912, 0.024776906, -0.0460931, 0.004281139, 0.020152679, 0.014573772, -0.0036956521, -0.038604837, 0.0010656232, 0.008465318, -0.033533104, 0.026566928, -0.023508972, 0.008353442, 0.022151537, -0.039410345, 0.03046023, 0.0012781885, -0.0044675996, 0.013410258, 0.009032159, -0.069691576, -0.035173956, -0.027118852, -0.02152503, 0.030609397, -0.044690914, 0.023881894, -0.04424341, -0.008465318, -0.026924934, -0.017422894, 0.00641425, 0.013716053, 0.014290352, -0.0041506165, -0.031892247, -0.022658711, -0.016811302, 0.0073726582, 0.022330541, 0.04006669, 0.032846928, -0.045406923, -0.06181547, -0.030117141, 0.07195894, 0.054625545, 0.020182513, -0.060801122, 0.0029143817, 0.012358619, -0.016885886, -0.026149256, 0.022300707, 0.031295575, -0.0028211512, 0.051910676, -0.00043165666, -0.014506647, -0.01772123, -0.012477954, -0.013566885, 0.04176721, 0.035054624, 0.009464748, -0.020257097, 0.00039086837, -0.009434914, -0.015558286, 0.0566244, 0.034577284, 0.015767122, 0.013141754, -0.028580705, -0.015841706, -2.8296877e-06, 0.013462467, 0.0073913042, -0.013507217, 0.021614531, -0.007037029, 0.021346027, -0.06993024, 0.0035147853, 0.0043557235, 0.029490633, -0.0066268155, 0.060651954, -0.007197385, -0.026447594, -0.031951915, -0.025403414, 0.0022226123, 0.032339755, -0.0037497259, 0.019496337, -0.030997237, -0.0043035145, 0.0039790724, -0.03720265, -0.008771114, 0.005772825, -0.00084466726, 0.041797042, -0.02243496, 0.043557234, 0.021704033, 0.059398938, 0.020510683, -0.018586408, -0.058354758, -0.030639231, -0.01487211, 0.033413768, 0.05620673, 0.029997807, 0.045973763, 0.020719519, 0.028133199, 0.021346027, 0.039649017, -0.016826218, -0.022464791, 0.019511255, 0.03767999, 0.032518756, 0.028237617, 0.041886546, 0.013059711, -0.07261528, -0.023628308, -0.02304655, -0.025194578, -0.01382793, -0.0077045583, -0.026432676, 0.051105164, -0.051582504, -0.01634888, 0.038425833, 0.037321985, -0.010993727, 0.0026906286, -0.016557716, 0.03207125, 0.05492388, -0.028327119, -0.031086737, 0.03046023, 0.00017469042, -0.040036853, -0.021181943, -0.054744877, 0.0106058875, -0.061994474, -0.03374194, 0.0038783837, -0.030669065, -0.030430395, -0.048897468, -0.032578424, 0.03511429, -0.010971351, -0.026029922, 0.07894003, -0.033413768, 0.08902382, 0.0017601895, 0.029087879, -0.04024569, -0.018079234, 0.01767648, 0.02376256, 0.04689861, -0.0004614904, 0.0016352609, 0.022494625, 0.0055005923, 0.017512394, 0.00046871573, -0.014670732, -0.050568156, -0.0070109246, -0.033294432, 0.018586408, 0.03588997, 0.022882463, -0.030102225, -0.004683894, 0.014118808, 0.04024569, 0.050717328, -0.030609397, 0.09027684, 0.019347169, -0.014543939, 0.016557716, 0.006045058, -0.027402272, -0.020898521, 0.016483132, -0.019660423, 0.05012065, -0.03863467, -0.015841706, 0.0012986992, -0.00551178, 0.024418902, 0.024970824, -0.0018058724, -0.018079234, -0.009039617, 0.02741719, -0.023553723, 0.016214628, 0.02570175, -0.023508972, 0.021480279, -0.0050046067, -0.0538797, 0.002280415, 0.034845788, -0.027626026, 0.029580135, 0.011769403, 0.038425833, 0.025269162, 0.051851008, 0.012321327, -0.008674154, 0.013641469, 0.023792394, 0.01335059, -0.042065546, -0.037709825, -0.067722544, -0.008539903, 0.004598122, 0.02802878, -0.05349186, -0.0064030625, 0.0010050235, -0.060323786, -0.041260038, -0.009084368, -0.008271399, -0.01876541, -0.015990874, -0.009628833, 0.03150441, -0.009091826, 0.0015746611, 0.0044452245, -0.002918111, 0.034726452, -0.0041133245, 0.008114772, 0.007928312, 0.024075814, -0.03368227, 0.024060896, -0.023031633, 0.001672553, -0.0012129273, -0.0053775283, -0.02798403, 0.0024482298, -0.030236477, 0.030579565, -0.008345984, 0.017467644, -0.0107998075, -0.023881894, 0.028595623, -0.0053476947, -0.0016622976, 0.01952617, -0.0149690695, -0.036128636, -0.005921994, 0.013932348, 0.00055099156, 0.009651209, 0.014573772, 0.018407406, 0.030072391, -0.007089238, -0.0005626453, -0.015722372, -0.040902033, -0.0068542976, 0.015140614, -0.027372438, -0.03102707, 0.011769403, 0.02295705, -0.011769403, 0.021241609, 0.030639231, -0.0053812573, 0.003222042, 0.00029204416, -0.037142985, -0.010173299, 0.004717457, -0.0075031808, -0.021002939, -0.013992015, 0.021256527, -0.009725793, -0.03383144, 0.011791779, -0.045824595, 0.025865836, -0.026924934, -0.010188215, 0.014081516, -0.008532444, -0.025075242, 0.0149466945, -0.042453386, -0.022837713, -0.013805554, -0.0054148203, -0.016125126, -0.0058287634, 0.052000176, -0.01591629, 0.015841706, 0.018899662, 0.013932348, 0.0017937525, 0.0013238714, 0.01030755, -0.020868689, -0.02452332, 0.020540517, 0.028148117, -0.034577284, 0.00040438678, 0.0150809465, 0.016572632, 0.00070948317, 0.025552582, 0.012194534, 0.020823937, -0.0004283936, -0.013760804, -0.004922564, -0.011724653, 0.003016935, -0.03872417, -0.036755145, 0.0022393938, -0.0141561, 0.009822752, 0.0010590971, -0.042662222, -0.04305006, -0.040424693, 0.07941736, 0.0015830519, 0.016169878, -0.015722372, -0.021167025, -0.027014434, -0.023106217, -0.014693107, -0.011113062, 0.01767648, -0.04913614, -0.043079894, -0.0013704866, 0.042274382, 0.017363226, 0.03159391, 0.0029013294, -0.0057802834, -0.04242355, 0.020838855, 0.006619357, -0.015453868, -0.027685693, -0.007816435, 0.0017051836, -0.047107447, 0.0012474224, 0.022554293, -0.005817576, 0.023911728, -0.05280569, -0.048151627, 0.01691572, -0.013820471, 0.0049486686, -0.021226693, 0.025119994, -0.009770543, -0.008472777, 0.01463344, 0.015364367, -0.0027931822, -0.012321327, -0.027059184, -0.005015794, 0.029281797, -0.013186505, -0.0538797, -0.013365507, 0.019302418, 0.008129689, -0.023389637, -0.008390734, -0.042304218, -0.02641776, 0.063963495, 0.013000044, -0.035770632, -0.014708024, -0.030758567, 0.0039790724, 0.02080902, -0.010397051, -0.011866363, -0.006306103, -0.02361339, 0.008592112, -0.0071190717, -0.010523845, 0.012761374, -0.0230018, 0.0064701885, 0.019198, -0.02746194, -0.01872066, -0.0106058875, -0.027402272, 0.023896812, 0.015334533, 0.012298952, 0.008510069, -0.014185934, -0.0010115496, 0.01980959, -0.032011583, 0.0027148686, -0.006183039, -0.015424034, 0.038187165, -0.020227263, 0.014208309, -0.002489251, 0.005907077, 0.016333962, -0.00803273, 0.024881324, -0.013358049, -0.023792394, -0.0150063615, 0.014432062, -0.014708024, -0.012798666, 0.025060326, 0.009009784, 0.022375291, -0.013425174, 0.0061233714, 0.021793533, 0.012313869, 0.01425306, -0.009278287, 0.015767122, -0.015259949, -0.003563265, -0.02541833, 0.012761374, 0.005992849, 0.011911113, 0.005164963, 0.018302988, 0.0075628483, 0.01520774, 0.053193524, -0.033324268, 0.055222217, -0.0034644408, -0.047853287, -0.001061894, 0.009658667, 0.021241609, -0.027521608, 0.0023344888, -0.006134559, 0.012388453, -0.024314484, -0.026208924, -0.013648927, -0.039499845, -0.0065410435, -0.016378714, 0.022405125, -0.016796386, -0.016975388, 0.020152679, -0.0022748213, -0.12166192, 0.0011318168, 0.007943228, 0.0149094025, -0.0033469705, -0.012738999, 0.04755495, 0.0033898565, -0.019451587, 0.013387882, 0.0034271488, -0.022136621, 0.018123986, -0.013432633, 0.022106787, -0.066827536, -0.0017751064, -0.03254859, 0.020436099, -0.028580705, -0.013164129, 0.021331111, 0.0014571908, -0.006813276, -0.0410512, 6.205647e-05, -0.011814154, -0.0015308429, -0.008584653, -0.025597332, -0.0151704475, 0.036277805, 0.0046540606, 0.021554863, 0.013455008, -0.0105536785, -0.0059369104, 0.00031465254, 0.02171895, 0.00088801933, -0.00883824, -0.027163602, 0.004732374, -0.03711315, 0.021644365, -0.0036080156, -0.0056534903, -0.0027913176, 0.024180232, 0.007637433, -0.009576624, -0.0022785505, -0.023136051, 0.0035949633, 0.022599043, -0.0062799985, 0.008808406, 0.001537369, 0.02850612, 0.03541263, 0.0039044882, 0.035382792, 0.031116571, -0.01382793, 0.0055677183, -0.009546791, 0.035382792, 0.02855087, -0.023300136, -0.023643224, -0.032518756, 0.009084368, 0.0150809465, 0.01534945, -0.023822226, -0.015200281, 0.043557234, 0.042184882, -0.01904883, -0.009315579, 0.0073279077, -0.011239855, -0.018631158, 0.030117141, -0.0047733956, 0.02589567, -0.0046018516, -0.02726802, 0.006410521, 0.04555609, 0.026820516, -0.019078664, -0.0030038827, 0.0067386916, -0.04448208, -0.0268951, -0.011776862, 0.0021815908, 0.02594042, 0.011575484, 0.029237047, 0.042542886, -0.021704033, -0.036755145, -0.009129119, 0.036665644, 0.0017723095, -0.009449831, -0.025492914, 0.0025097618, 0.012269118, 0.0066491906, 0.00821919, 0.021644365, -0.01297021, 0.023732726, 0.049673147, -0.0053327777, -0.017989734, -0.004351994, 0.017348308, -0.005150046, -0.004504892, 0.007298074, -0.0015662704, 0.00779406, -0.012463037, 0.012597289, 0.013231255, 0.0032984908, 0.007868644, 0.012313869, -0.0028584434, -0.0007537676, 0.006485105, -0.006328478, -0.003787018, -0.0019298688, -0.031444743, 0.018780328, -0.005869785, -0.0149317775, 0.014506647, 0.0015112645, -0.020122845, -0.036039136, -0.042751722, -0.00978546, 0.008428026, 0.022270873, 0.042930726, 0.0011392753, -0.011314439, -0.0047099986, -0.034278944, -0.010993727, -0.006496293, -0.010046505, -0.0017601895, 0.030176809, 0.038843505, 0.04209538, 0.045078754, -0.0015187229, 0.019675339, -0.01606546, -0.0015327075, -0.012463037, -0.011202563, 0.015647788, -0.06521651, 0.00422893, 0.012425745, 0.049404643, 0.067006536, 0.015811872, 0.0020939542, 0.018422322, 0.018183652, 0.033085596, 0.038992673, 0.03806783, -0.020257097, 0.0027745361, 0.012664415, -0.025343746, 0.0052432767, 0.004105866, 0.023896812, -0.023165885, -0.016408546, 0.026402844, 0.0015718642, 0.012269118, -0.009569166, 0.017422894, 0.01667705, -0.0069251526, -0.03177291, -0.004191638, -0.02883429, 0.02028693, -0.007913395, -0.0010861339, 0.0036881939, 0.008614487, -0.012627123, 0.014036766, -0.001743408, 0.019943843, -0.024791822, -0.019690257, 0.0034644408, 0.011269689, -0.05507305, 0.008897907, 0.014409687, -0.008010354, -0.025552582, -0.011202563, -0.05668407, -0.028013865, -0.017258808, 0.017810732, -0.0107028475, -0.015274866, 0.03550213, -0.011843988, -0.024210066, 0.010441802, -0.021420611, -8.862713e-05, -0.0026328259, 0.0072906157, -0.014066599, 0.014208309, -0.047763787, -0.0024295836, -0.0051761507, -0.007965604, 0.04800246, 0.023300136, -5.400807e-06, -0.024016146, -3.962291e-05, -0.033562936, 0.0042513055, 0.0047920416, -0.00058595295, -0.015245032, -0.007301803, 0.016319046, 0.009285746, -0.017422894, 0.016751634, -0.03854517, 0.029520467, 0.028565789, 0.024031062, 0.015975958, 0.020749353, 0.04376607, -0.004035011, -0.022151537, 0.0005784945, -0.011829071, -0.0075814943, 0.012112491, 0.030206643, -0.031325407, 0.0150809465, 0.028207783, -0.0051612337, 0.033383936, -0.029938139, -0.001432951, 0.0008549226, -0.019391919, 0.01776598, -1.1428005e-05, -0.0074584302, -0.008368359, -0.009628833, 0.009211161, -0.021614531, -0.025686834, -0.029237047, 0.0066678366, 0.015409117, -0.008122231, 0.002532137, 0.0037609134, -0.00560874, 0.010091256, -0.030236477, -0.0013639604, -0.012209451, 0.023270303, 0.03526346, -0.048927303, -0.014387312, -0.006988549, 0.0014534616, -0.025030492, 0.0016595008, 0.032011583, 0.0411407, -0.04537709, -0.006693941, -0.0002587143, -0.015148072, 0.03421928, -0.0037553196, -0.0012931053, 0.0053066732, -0.006343395, -0.012656956, 0.0073428247, -0.00855482, -0.022792963, -0.030117141, 0.0011103738, -0.021853201, 0.022643795, -0.01396964, -0.019362085, 0.01815382, -0.00054912694, -0.0047398326, 0.02509016, 0.029207213, -0.0192875, 0.050389156, 0.0005258193, -0.026775764, 0.023195717, -0.04075286, -0.0150138205, -0.00021093374, -0.0007849998, 0.015289783, 0.0055826353, 0.004422849, -0.011635152, 0.01164261, 0.011553109, 0.013298381, -0.011955864, 0.023538806, 0.020257097, 0.084608436, -0.022136621, -0.042871058, -0.013716053, -0.01230641, 0.012246743, -0.022986881, -0.004143158, -0.035233624, -0.019466503, -0.016095292, 0.03168341, 0.016990304, 0.007163822, 0.019973677, 0.0061457464, -0.0072085727, -0.0039417804, 0.04722678, 0.00418045, 0.0064776465, 0.0537902, 0.003971614, -0.008592112, 0.0102628, 0.0243294, 0.021346027, 0.008868074, -0.0025097618, 0.03920151, -0.009651209, -0.04000702, -0.021196859, 0.0056534903, -0.035561796, 0.01382793, 0.0014683785, 0.02746194, -0.034607116, 0.022599043, -0.009539332, -0.009845128, 0.017258808, 0.012090116, 0.029997807, 0.0055155093, 0.011038477, -0.00011175992, 0.011903655, -0.016557716, -0.021152109, -0.00912166, -0.0115009, -0.018974246, 0.03896284, -0.04066336, 0.0014599877, 0.029803887, 0.01972009, 0.016498048, -0.036904313, -0.0029348924, -0.03213092, 0.03254859, 0.0024668758, -0.07840302, -0.010934059, 0.0007770752, 0.0105984295, 0.045168255, -0.026253674, -0.018422322, 0.0013210744, 0.003801935, 0.006656649, 0.00037082384, -0.018019568, -0.00888299, 0.006969903, 0.026328258, -0.015901374, -0.026910016, -0.026865266, -0.026432676, 0.000650748, -2.5856863e-05, 0.048062123, 0.028923793, -0.029609969, 0.008823323, -0.032369588, 0.03630764, 0.0411407, 0.077209674, 0.021062607, -0.018138902, -0.0030560917, -0.004493704, 0.0049523977, -0.0008404719, 0.020734437, -0.0010180757, -0.036337472, 0.0021275172, -0.013380424, -0.031146405, -0.026387926, 0.00028015728, 0.018869828, 0.023628308, -0.0067498796, 0.0034532533, 0.0106730135, 0.003382398, -0.020346599, 0.023061467, 0.0015625411, 0.03263809, -0.007055675, -0.038187165, 0.0167218, 0.025836002, -0.0038746546, -0.017094722, -0.009964463, -0.024597904, 0.014611065, 0.008927741, 0.0047398326, -0.012955293, -0.009755627, 0.01088185, 0.01772123, 0.0019084258, 0.03735182, -0.011485983, -0.019183083, -0.008249024, -0.006902777, 0.03320493, -0.0051761507, 0.008621945, -0.038783837, -0.019645505, 0.0061271004, 0.0042923265, 0.0067386916, -0.006119642, 0.009986838, 0.019451587, -0.018869828, -0.035144124, 0.028998377, -0.017646646, 0.0037665074, -0.027208354, 0.0026682534, -0.006775984, -0.0038933007, -0.020734437, 0.03970868, 0.0047920416, -0.03654631, 0.013909972, 0.026044838, 0.010859475, -0.018914578, 0.0032649278, 0.023538806, 0.017094722, -0.0078089763, 0.015573203, -0.001000362, 0.014387312, 0.00389703, -0.01739306, 0.01776598, -0.0006251097, -0.004243847, 0.035084456, 0.0042736805, -0.021808451, 0.004389286, -0.005526697, -0.007965604, 0.01819857, 0.005168692, 0.0037944764, -0.0065410435, -9.521152e-05, -0.0037926119, -0.022390207, -0.004385557, 0.007764226, 0.032429256, 0.020883605, -0.03911201, 0.015073488, 0.01819857, 0.0106282635, -0.007950687, 0.009710876, 0.0043743695, 0.009442373, -0.02770061, 0.021927785, -0.017512394, 0.017079806, 0.0065820646, 0.0044079325, -0.021793533, -0.0040126354, -0.01453648, -0.0063172905, 0.034159612, -0.030609397, 0.0009341684, -0.0052581932, -0.020167595, -0.013857763, 0.02585092, 0.007987979, 0.007928312, 0.013134296, -0.027491774, 0.013298381, 0.0027913176, -0.017661562, 0.039798185, 0.016304128, -0.0041021365, -0.030102225, -0.007823894, 0.017825648, -0.00095094985, -0.0074136797, -0.028088449, -0.01387268, -0.004299785, 0.007950687, -0.008651779, 0.029416049, -0.0030784672, 0.035472296, -0.013648927, 0.05602773, -0.011955864, -0.009158952, -0.011605318, -0.01401439, 0.024075814, 0.03207125, -0.03976835, -0.029923223, -0.007913395, -0.04328873, -0.0042550345, -0.017616812, -0.025119994, 0.0053700698, -0.031325407, -0.009576624, 0.016125126, 0.008502611, 0.008532444, 0.0150884045, 0.02076427, 0.005992849, -0.01112052, 0.011918572, -0.017885316, -0.0038131224, -0.038933005, -0.033891108, 0.015990874, 0.050150484, 0.0039380514, 0.023374721, 0.016095292, -0.009994296, 0.013939806, -0.014767692, -0.012530163, -0.0128285, -0.012194534, 0.031862415, 0.018452156, 0.0032817093, -0.031951915, -0.013216338, 0.015856624, 0.020734437, 0.029908305, -0.010456719, 0.00027689422, 0.00165111, -0.007779143, -0.021510113, -0.0105014695, 0.013059711, -0.028148117, -0.019227833, 0.014267977, -0.009964463, 0.015230115, 0.012642039, -0.008584653, 0.03759049, 0.00874128, -0.009539332, 0.023210635, -0.0067834426, -0.002455688, -0.0025973984, 0.025806168, 0.025746502, 0.03281709, -0.0074285967, -0.035084456, 0.0038933007, -0.010098714, 0.036098804, 0.0074733472, 0.0051985257, 0.015975958, -0.011776862, -0.009233537, 0.0074733472, 0.019108498, -0.009032159, 0.030042557, 0.0064552715, 0.0015177906, -0.01016584, -0.0059555564, -0.034278944, -0.010173299, -0.007387575, 0.021554863, -0.0069065066, 0.008987408, -0.017228974, -0.031713247, 0.006086079, -0.020958189, 0.044183742, -0.036367305, -0.00057616376, -0.013052253, -0.0017378143, 0.022032203, 0.02931163, 0.0012912408, -0.038604837, -0.014722941, 0.005183609, 0.015245032, 0.021480279, 0.015931208, -0.034815952, -0.057937086, -0.006600711, 0.03135524, 0.001525249, -0.010150923, 0.0134923, -0.0067200456, -0.05384987, -0.0055192383, 0.006846839, 0.00095141603, 0.018302988, 0.034159612, -0.007339095, -0.032667924, 0.01534945, -0.030057475, -0.008965033, 0.02532883, -0.020823937, 0.02509016, -0.03174308, 0.04075286, -0.015424034, 0.027894529, -0.025836002, -0.023225551, 0.017870398, 0.020107929], "content_hash": "fedcd60a1e79952b153140d3e9539021"}, {"content": "# Experiment: Comparing Heads Across Categories\n\n## Goal\n\nSystematically ablate heads from each category to discover which types of attention heads matter most for different prompts. Build intuition for how attention head roles vary.\n\n## Prerequisites\n\n- Complete \"Your First Ablation\" (know how to use the ablation panel)\n- Complete \"Exploring Attention Patterns\" (understand head categories)\n\n## Steps\n\n### Step 1: Set Up a Simple Prompt\n\n1. Select **GPT-2 (124M)** and enter: `The cat sat on the`\n2. Set beams to 1 and tokens to 5.\n3. Click **Analyze**.\n4. **Select the generated sequence for comparison** by clicking \"Select for Comparison.\"\n5. Note the original prediction and probability in Stage 5 (e.g., \"mat\" at 45%).\n\n### Step 2: Record the Head Categories\n\n1. Expand **Stage 3 (Attention)** and note one head from each category:\n   - **Previous-Token**: _______ (e.g., L0-H3)\n   - **First/Positional**: _______ (e.g., L0-H1)\n   - **Bag-of-Words**: _______ (e.g., L2-H5)\n   - **Syntactic**: _______ (e.g., L4-H2)\n   - **Other**: _______ (e.g., L1-H8)\n\n### Step 3: Ablate One Head at a Time\n\nFor each head you noted, do the following:\n1. Go to the **Ablation** tab in the Investigation Panel.\n2. **Clear** any previously selected heads.\n3. **Add** just the one head from the current category.\n4. Click **\"Run Ablation Experiment.\"**\n5. Record the results:\n\n| Category | Head | Probability Change | Generation Changed? |\n|----------|------|-------------------|-------------------|\n| Previous-Token | | | |\n| First/Positional | | | |\n| Bag-of-Words | | | |\n| Syntactic | | | |\n| Other | | | |\n\n### Step 4: Analyze Your Results", "source_file": "experiment_comparing_heads.md", "chunk_index": 0, "embedding": [0.013296178, 0.01098963, 0.027249454, 0.0021908856, -0.031433422, -0.030360611, -0.016735885, 0.02944872, 0.026833737, 0.044521745, 0.052031435, -0.036368363, 0.00055694085, 0.0039559985, 0.02197926, -0.0039392356, 0.03610016, -0.008354532, -0.0005695129, 0.06769451, -0.011063385, -0.013436984, 0.01956543, 0.047257416, 0.022368154, -0.017553905, 0.020584602, 0.039667264, 0.025801156, 0.010486748, 0.02789314, -0.005062337, -0.029019594, 0.01968612, -0.038996756, 0.0015011003, -0.005504872, 0.02141603, -0.0098028295, -0.0030625463, 0.016883396, -0.04575548, -0.047096495, 0.012209954, -0.028751392, 0.039640445, 0.055893563, -0.013671662, -0.01882787, 0.017299112, -0.022287693, 0.06270593, -0.025640232, 0.017138189, 0.01625312, 0.013698482, -0.018935151, -0.0030893665, 0.017017499, -0.008723311, -0.0036777374, -0.006601153, 0.05503531, 0.000619801, -0.0008050289, -0.032479417, -0.046935573, 0.015823994, 0.05259466, 0.015059614, 0.034544583, 0.023655528, 0.01104327, 0.0058401264, 0.005746255, -0.031567525, 0.0069196443, -0.0032888427, -0.010768361, -0.03859445, 0.02786632, 0.022274282, 0.053640656, -0.017755058, -0.009400525, -0.032345314, -0.03864809, -0.038809013, -0.02620346, -0.060721222, -0.0063161873, 0.017741647, -0.043073446, 0.011418754, -0.013604611, -0.011163961, 0.037816662, -0.030119227, -0.0009202725, 0.026847148, 0.04527271, -0.0054814043, -0.043609854, -0.0065173395, 0.008857413, -0.0051360927, -0.0076572034, 0.015059614, -0.033310845, -0.02613641, -0.03137978, -0.0039995816, -0.0069129393, 0.023843272, 0.021201469, -0.0010736512, -0.03682431, -0.0040934524, 0.028000422, -0.06903552, -0.019417917, -0.003097748, 0.016065376, -0.0046533267, 0.017500265, 0.0005862756, 0.04830341, -0.056590892, 0.0014340493, -0.01953861, 0.04489723, 0.018666947, -0.0072683087, -0.043985337, -0.020101836, 0.013309588, -0.02450037, 0.048062027, -0.06946465, 0.021469673, -0.0026870617, 0.012028917, -0.012049032, 0.042242017, -0.06334961, -0.011988686, 0.012176429, 0.07498964, -0.031889368, 0.042751603, 0.010412992, 0.01389293, -0.010044212, -0.032345314, -0.008890939, -0.06463699, -0.004308015, -0.0017315875, -0.029207338, -0.07799351, 0.0734877, -0.02293138, -0.0035436358, -0.016507912, -0.03768256, 0.031299323, -0.034088638, 0.010453222, -0.03540283, -0.011532741, -0.048517972, -0.033391308, 0.017138189, -0.026780097, -0.0024959669, 0.03355223, -0.04580912, -0.0040196967, -0.014389106, -0.028134523, 0.0132224215, 0.01271954, -0.016145837, 0.006581038, -0.018465795, -0.044736307, 0.0149255125, 0.002708853, -0.04358303, -0.04006957, 0.028429547, 0.0030139342, 0.057502784, 0.025720695, -0.017701415, -0.026659405, 0.057931907, 0.034893245, 0.035724677, -0.009762599, 0.016829755, -0.0022344685, 0.015019383, 0.001239602, 0.02385668, 0.027464015, -0.014429336, 0.00050665275, -0.049698066, -0.030494712, 0.04586276, -0.07466779, 0.012203249, 0.03685113, -0.073058575, -0.02706171, -0.06624621, 0.034598224, 0.023266634, 0.020758934, 0.05809283, -0.02950236, 0.045540918, 0.048866637, -0.022891149, 0.018479206, 0.060292095, -0.024781983, 0.002733997, -0.04151787, 0.014308644, 0.024111474, -0.014228184, 0.012095967, -0.037870303, -0.04924212, 0.052138716, -0.003414563, 0.041249663, -0.02696784, 0.007704139, 0.0009403877, 0.022167, -0.02781268, 0.02773222, 0.003520168, -0.015233946, -0.06334961, 0.0096217925, 0.04178607, 0.06592437, -0.0010987953, -0.048625253, 0.011572971, -0.018733999, 0.0016335256, -0.00045971718, 0.0017718179, -0.027075121, 0.0055853333, -0.04910802, 0.013027974, -0.016199477, 0.035912417, -0.021778107, -0.038969938, -0.023239814, -0.0074225254, -0.051226825, -0.026659405, -0.06635349, 0.003607334, 0.037199795, 0.010439812, -0.028134523, -0.005018754, -0.0033575697, -0.024486959, -0.029126875, -0.02200608, -0.020852804, -0.0014072291, 0.025827976, -0.0009965427, -0.017728237, 0.008689786, -0.0056423265, 0.00015149295, 0.013564381, -0.02613641, 0.014630489, -0.052004617, 0.06393966, -0.04253704, -0.016615193, 0.015180306, 0.0106610805, 0.021429442, -0.017875748, 0.050180834, 0.03355223, -0.023038661, 0.05482075, 0.0002820325, 0.031406604, 0.020960085, 0.02294479, 0.0019562077, -0.002732321, 0.023762811, -0.01101645, -0.013209011, 0.009722369, -0.021348981, -0.040498696, -0.03202347, -0.0014164485, -0.010922578, -0.008381353, -0.004774018, -0.06281321, -0.04189335, 0.010533684, 0.03119204, 0.0237494, 0.01884128, -0.030682454, 0.005645679, 0.0031497122, -0.025170878, 0.03030697, -0.015140075, 0.0085758, 0.018358514, -0.018506026, 0.009226193, -0.024902673, 0.07134207, -0.007127502, 0.048062027, -0.0426175, 0.018197592, -0.0155289695, 0.012598849, -0.030870197, 0.057824627, 0.011009744, -0.007865061, 0.044602204, -0.0007865899, -0.044387642, 0.017728237, -0.008079624, -0.0044085914, -0.012109378, 0.05077088, -0.02958282, -0.045460455, 0.015180306, 0.012585439, 0.028965954, 0.023655528, 0.019122893, 0.010392877, 0.034490943, -0.019122893, -0.03864809, 0.002752436, 0.01880105, 0.0024775278, -0.0003545312, -0.016869986, -0.0065408074, -0.041222844, -0.0674263, -0.025680464, -0.022730228, -0.020155476, -0.0017005764, 0.047176957, -0.018211002, -0.02950236, -0.041196022, -0.06292049, 0.033793613, 0.029368259, 0.019806812, 0.00811315, -0.07734983, -0.0035302257, 0.011244423, -0.014670719, -0.007838241, 0.026149819, -0.024057833, 0.05428434, -0.024916084, 0.061525833, 0.038835835, 0.06533432, 0.014295234, 0.009628498, -0.041303303, -0.011539446, 0.034330018, 0.0056557367, 0.018975383, 0.007898587, 0.009776009, 0.0030306971, 0.06495883, -0.0046298592, 0.043851234, 0.009963752, -0.0072414884, 0.014858462, 0.019914092, 0.020638242, -0.000861603, -0.012404402, -0.008441698, -0.06823091, -0.010768361, -0.023910321, -0.03690477, -0.014617079, 0.02456742, -0.01969953, 0.045943223, -0.038353067, -0.0035134628, 0.06383238, 0.04151787, -0.009648613, -0.01802326, -0.0072280783, 0.0025261396, 0.063993305, -0.026900789, -0.019243585, 0.014335465, -0.017325932, 0.02385668, 0.010433108, -0.046935573, 0.012793296, -0.013135255, -0.08915077, 0.02610959, -0.029797385, -0.030441072, -0.007529807, -0.0073219496, 0.033284027, 0.013088319, -0.023494607, 0.02952918, 0.008367943, 0.095855854, -0.009226193, 0.018760819, -0.022555895, -0.008019278, 0.019337457, -0.0053338925, 0.040579155, 0.010929284, -0.024339447, 0.021697644, -0.018304873, 0.044575386, -0.0022294398, 0.0053439504, -0.028590469, -0.030709276, -0.012203249, -0.004995286, 0.026002308, 0.0048779473, -0.053882036, 0.030682454, -0.023105713, 0.034222737, 0.049698066, -0.0038520696, 0.04342211, -0.012411106, 0.0039895237, 0.016910216, 0.010734837, -0.0024875854, -0.006866004, 0.010319121, -0.015005973, 0.03419592, -0.042349298, -0.0037615509, 0.019525198, 0.0044722897, -0.004026402, 0.026833737, -0.033874072, 0.010506864, 0.051924154, 0.0067821904, 0.014912102, 0.05551808, 0.013108435, -0.027128762, 0.040847357, -0.030655634, -0.0007023573, -0.008662966, 0.014228184, -0.01640063, 0.026324151, -0.00066967006, 0.01969953, 0.016145837, 0.014147723, -0.025036776, -0.0038822424, -0.03255988, 0.0013963333, 0.0011432165, -0.009360295, -0.01963248, -0.056912735, -0.04015003, -0.019793402, 0.004220849, -0.030950658, 0.002658565, 0.01799644, -0.034920067, 0.0008968047, 0.01644086, -0.005682557, -0.03926496, -0.0047203777, 0.008528864, 0.017661186, 0.012779886, -0.01634699, 0.02294479, -0.008012573, 0.063778736, -0.00617538, 0.016467681, 0.02706171, -0.013537561, -0.03028015, 0.021496493, -0.01394657, 0.0018573077, -0.0065910956, -0.037226614, -0.04186653, -0.006470404, -0.025546363, 0.028939134, -0.046399165, 0.01192834, -0.015931275, -0.00019706655, 0.021912208, 0.002925092, 0.014322055, 0.03186255, -0.012652489, 0.008066214, -0.017553905, 0.020088425, -0.030494712, 0.028885493, 0.02041027, -0.02301184, 0.015703302, 0.0098028295, -0.01476459, -0.022676587, -0.01063426, -0.02209995, 0.049805347, 0.00026003146, -0.042885702, 0.025184287, 0.009615088, -0.030119227, -0.024124885, 0.016802935, -0.031031119, 0.021348981, 0.01880105, -0.022609536, -0.004790781, 0.024661291, 0.009206077, -0.0078047155, -0.0004096386, 0.021871977, 0.016025146, -0.00224285, 0.03030697, -0.028054062, 0.0067788376, -0.028778212, -0.0046935575, 0.01645427, 0.029985126, 0.008723311, 0.0077242544, -0.0490812, -0.013215716, 0.013296178, 0.013732008, -0.0060647465, 0.012531798, 0.007992458, -0.028939134, 0.00041906763, -0.008381353, -0.021939028, 0.012370876, 0.016869986, 0.010533684, 0.022368154, -0.031701628, 0.033176746, 0.003329073, -0.047793824, 0.029073235, 0.018452385, 0.020115245, -0.0085758, 0.008797067, -0.008904349, 0.0070202206, 0.02372258, 0.012813412, -0.014912102, 0.002504348, -0.010929284, -0.016199477, -0.01471095, -0.0051226825, 0.006654794, -0.015676482, -0.03347177, -0.012867052, -0.010949398, -0.027329914, 0.07187848, 0.0079187015, 0.025144057, -0.0024121532, -0.01634699, -0.01228371, -0.025197698, 0.0048779473, -0.03261352, 0.007938817, -0.008321007, -0.020370038, 0.01150592, -0.008059508, -0.0137051875, 0.03202347, -0.016574962, 0.006513987, -0.05398932, 0.01720524, 0.041651968, 0.007040336, 0.010325826, 0.04819613, 0.0029016242, -0.0063295974, -0.007529807, 0.0069196443, -0.02531839, 0.027155582, -0.04900074, -0.014912102, 0.015354638, -0.01889492, 0.008522159, -0.04436082, 0.0002348874, 0.014013621, -0.0081600845, -0.044521745, 0.0020500787, -0.005052279, -0.0057361973, 0.0013334731, -0.024688112, 0.03030697, -0.014858462, -0.09880608, -0.03939906, -0.0098028295, 0.017902568, 0.020007964, -0.00346988, -0.0115796765, 0.01146569, 0.07193212, 0.01102986, -0.02610959, -0.031406604, 0.003129597, 0.0049483506, 0.036985233, -0.017071139, 0.03942588, -0.024956314, -0.0070604514, -0.008911054, -0.033230387, -0.036770668, 0.026726456, -0.001555579, 0.021456262, 0.027195811, -0.018465795, -0.0043884763, -0.022381563, -0.014885282, 0.001623468, 0.041222844, 0.033874072, 0.025801156, -0.054525726, -0.014255004, 0.037092514, -0.027195811, -0.003086014, 0.023494607, -0.016494501, 0.049322583, -0.0048108962, 0.02629733, 0.01142546, 0.020209117, -0.001183447, -0.010875643, 0.014563438, -0.03347177, -0.018157361, -0.022314513, 0.0118210595, -0.0089579895, -0.011184077, 0.023762811, -0.0037079102, 0.0031262445, -0.03344495, -0.007999163, 0.046157785, 0.02141603, -0.0030826614, -0.007637088, -0.007147617, -0.03754846, -0.049858987, 0.036180623, -0.020866215, 0.0013250918, 0.023963964, 0.008213725, 0.008522159, 0.033203565, -0.01387952, 0.04039141, -0.03620744, 0.036395185, -0.015488739, -0.018090311, 0.028751392, 0.012411106, 0.007100682, -0.056108125, -0.022515666, -0.013450394, 0.0012613935, -0.021147829, -0.01634699, -0.014308644, -0.03290854, -0.00812656, -0.0010426402, 0.015998326, -0.01391975, -0.003992876, -0.003414563, -0.019122893, -0.11285994, -0.011090205, 0.015086434, -0.026055949, -0.04495087, 0.0027876378, 0.02947554, 0.015756942, 0.017004088, 0.015971504, 0.020477321, -0.030146047, 0.00022545838, -0.012893872, 0.025640232, -0.038165327, 0.007992458, -0.017527085, 0.013336408, -0.03137978, -0.025787745, 0.026310742, -0.0016787848, -0.0053841807, -0.056322686, 0.007207963, -0.03019969, 0.0027759038, 0.005508225, 0.034705505, -0.032050293, 0.017795287, -0.045460455, 0.009615088, -0.02624369, 0.035080988, -0.0078114206, 0.015743533, 0.011814354, -0.011955161, 0.014469567, -0.0058434787, -0.007838241, -0.014322055, 0.02539885, -0.011780829, -0.0058166585, -0.0048779473, 0.01269272, 0.035510115, 0.0064905193, 0.003848717, 0.009594972, 0.022891149, 0.03494689, -0.024768572, -0.0008310111, -0.02057119, 0.023400735, 0.03274762, 0.018170772, 0.028885493, 0.05221918, -0.016816346, 2.4449977e-05, 0.010037508, -0.009749189, 0.03376679, -0.015193716, -0.0327208, -0.022354743, 0.0026853853, -0.0029301208, 0.00024452596, -0.0060077533, -0.008763542, 0.01394657, 0.024916084, 0.0006654794, 0.0064603463, 0.051119544, -0.02456742, -0.017419802, 0.015381458, 0.0112041915, 0.0344373, 0.009628498, -0.00117758, 0.0058904146, 0.06415422, -0.0013435308, -0.03596606, -0.0022042957, 0.034732323, -0.036180623, -0.0015363019, 0.026498483, 0.012277005, 0.02054437, 0.019310636, 0.0056792046, 0.015368048, -0.00029523313, -0.024312627, -0.03280126, 0.0068056583, 0.012793296, 0.00554175, -0.015984915, -0.013235832, 0.04012321, 0.031674806, -0.017500265, 0.030709276, 0.003216763, 0.007844945, 0.047874283, 0.0021925617, 0.0083411215, -0.0031245681, -0.01559602, 0.02708853, -0.015260766, -0.012176429, -0.015716711, 0.031487066, -0.036502466, -0.012169723, 0.0038017815, 0.006899529, 0.052648302, 0.02041027, -0.020302989, -0.00939382, 0.018036671, -0.010010688, -0.013329702, -0.005769723, -0.014348875, 0.000502043, -0.0212417, 0.0030223157, 0.032157574, -0.0059708753, -0.0162263, -0.018680358, 0.0067218444, -0.014268414, -0.016883396, 0.014228184, 0.008394763, 0.006638031, -0.0031161867, -0.008978104, -0.013839289, 0.01810372, 0.02778586, 0.0032184394, -0.032640338, 0.022167, 0.01813054, 0.0019880568, 0.063671455, 0.021697644, 0.0035972765, 0.00010712416, -0.035671037, 0.0016385544, 0.020973496, 0.0028731276, -0.047096495, 0.028992774, 0.014831641, 0.041598327, 0.02197926, 0.027651757, -0.017366162, -0.02121488, -0.005162913, -0.0033827138, -0.017057728, 0.028778212, -0.01961907, -0.017661186, -0.0035536934, 0.014603669, 0.0026904142, 0.018304873, 0.01799644, -0.018036671, -0.0017081196, 0.023843272, -0.016320169, 0.0277054, -0.026149819, -0.016078787, 0.022864329, 0.0036039816, -0.0377362, -0.03288172, -0.037816662, 0.047257416, 0.0062927194, -0.009152437, 0.034678683, 0.040310953, -0.016735885, 0.015019383, -0.028054062, -0.018211002, -0.03690477, 0.020919856, -0.013537561, -0.0018522788, -0.043636672, 0.012250185, 0.0026350971, -0.007207963, 0.032345314, -0.011076795, -0.058521956, -0.016816346, -0.043019805, 0.022287693, 0.005169618, 0.015180306, 0.040740076, 0.0018103721, 0.019820223, 0.005075747, -0.03030697, -0.016642014, -0.014134313, 0.0050422214, -0.039962288, 0.017956208, -0.016239708, 0.0006759561, -0.00693976, 0.013322998, 0.03376679, 0.0155289695, -0.019337457, -0.0110835, 0.022837508, -0.022622947, 0.037816662, 0.018452385, -0.011251127, -0.010392877, -0.015005973, -0.012531798, 0.034330018, -0.008428288, 0.017486854, -0.01145228, 0.017446622, 0.013329702, 0.056215405, 0.010151494, -0.008381353, 0.018693767, -0.012451337, -0.05168277, 0.009782715, -0.0074024103, -0.016722474, -0.011908226, 0.013396754, -0.0075231018, 0.0114791, 0.015019383, -0.019189944, 0.023347095, -0.008810477, -0.01725888, 0.0081802, -0.010003982, -0.003216763, -0.034034997, 0.028831853, 0.021496493, -0.013564381, 0.007389, 0.0043616556, -0.01881446, -0.017594134, -0.009212783, -0.0013653223, -0.033391308, -0.00011335779, -0.013986801, 0.008012573, -0.021617183, -0.008515454, -0.003557046, -0.0033357781, 0.0027122055, 0.013396754, -0.029985126, 0.010433108, -6.746989e-05, 0.038218968, -0.02610959, -0.008227135, 0.029636461, 0.0026384497, -0.050180834, -0.010560504, 0.015086434, -0.0091993725, 0.011666843, 0.009863175, 0.03022651, -0.015005973, -0.03518827, 0.004096805, -0.018358514, -0.011184077, -0.042885702, -0.019887272, 0.035456475, -0.01872059, 0.0045695133, 0.002403772, -0.00674196, -0.0075566275, -0.008052804, -0.0032821377, -0.03355223, 0.0106610805, 0.020396858, -0.009836355, 0.00634636, -0.020530961, 0.026257101, -0.01709796, 0.00073420646, 0.034061816, 0.0043985336, -0.031433422, 0.003607334, -0.028214986, 0.0024071245, 0.019525198, -0.0025713989, -0.011700368, -0.020195706, 0.0059105298, 0.01969953, 0.08110467, -0.02621687, -0.047874283, 0.009889996, 0.004820954, 0.010459928, 0.0020919857, -0.0134638045, -0.046962395, -0.0102386605, -0.04028413, 0.03365951, 0.0016352019, 0.016145837, 0.004804191, 0.017111368, -0.036502466, -0.00089345215, 0.055839922, -0.009125616, 0.009548036, 0.055679, -0.038353067, 0.018961972, 0.026418023, 0.022703407, 0.023655528, 0.0006378209, 0.0011423783, 0.0009848089, -0.0051394454, -0.051709592, -0.034222737, -0.014617079, -0.023910321, 0.017808698, 0.0035000527, 0.014509797, -0.013618021, 0.0024691464, -0.015877634, 0.027222633, 0.023910321, -0.010801887, 0.0009764276, 0.011264537, -0.012578733, 0.017486854, 0.0009152437, -0.025184287, -0.028724572, 0.018465795, -0.006513987, -0.020450499, 0.036234263, -0.047150135, -0.015274176, 0.017325932, 0.011633317, 0.0034866426, -0.015944684, 0.042215195, -0.017755058, 0.00063823996, -0.0096217925, -0.03274762, -0.024419907, -0.0032184394, -0.006644736, 0.007134207, -0.032157574, -0.026096178, 0.005672499, -0.016789526, 0.011988686, 0.0026820328, -0.008012573, -0.015341227, 0.04146423, 0.008904349, 0.009353589, -0.021027137, -0.02452719, -0.039506342, -0.03019969, 0.0026451547, 0.039720904, -0.026485074, -0.048062027, 0.025975488, -0.029287798, 0.011968571, 0.03347177, 0.042912524, 0.012558619, -0.0073018344, -0.022287693, -0.011331588, 0.020584602, -0.02291797, 0.0015262442, 0.009212783, -0.013255947, 0.005742903, -0.049215302, 0.0009588267, -0.04167879, 0.018452385, 0.027464015, -0.002125511, 0.028510008, -0.009387115, -0.006376533, 0.01471095, -0.004126978, 0.01387952, -0.0060278685, 0.04819613, -0.033176746, -0.012625669, 0.008897644, -0.00984306, 0.015756942, 0.00054269255, -5.610268e-05, -0.0014298587, 0.03674385, -0.003674385, -0.013671662, 0.011727188, -0.01064767, 0.01394657, 0.02786632, -0.02291797, 0.014415926, 0.0036207442, 0.0039694086, -0.012163019, -0.044602204, 0.020142065, 0.0057965433, 0.01641404, -0.014617079, -0.027289683, 0.0015404925, 0.010587324, 0.0032502885, 0.006185438, 0.019766582, -0.016145837, -0.026350971, -0.025600003, 0.037414357, 0.0017517026, 0.0057294923, -0.018613307, -0.005716082, 0.012015507, 0.002281404, -0.010097853, 0.018438974, -0.000418439, -0.005917235, 0.0029703514, 0.017299112, 0.030011946, -0.032398954, -0.0013359876, 0.053774755, 0.013450394, 0.005974228, 0.0063932957, -0.0067319022, -0.003216763, -0.018519437, -0.010982924, -0.0016251443, 0.013571085, -0.004566161, 0.018935151, -0.011311473, -0.0089579895, -0.015837403, -0.025304979, -0.0072616036, 0.030011946, -0.00864285, -0.009098796, 0.0018656891, 0.022260873, -0.006011106, -0.023910321, -0.0008385543, -0.008528864, 0.0542307, 0.038755372, -0.020799164, 0.010640965, 0.028268626, 0.003724673, 0.01810372, -0.009735779, -0.0277054, -0.025103826, -0.019230176, 0.024889264, 0.013571085, -0.016829755, -0.0022026193, 0.009662023, -0.011532741, -0.0033659511, -0.013048089, -0.0029016242, -0.002336721, -0.010037508, 0.021603774, 0.0056993198, -0.01270613, -0.006688319, -0.0277054, 0.005323835, 0.05551808, 0.009527922, -0.021764696, 0.011867995, -0.028241806, -0.026645996, -0.011720483, -0.015958095, 0.011318178, -0.019498378, -0.011117026, 0.0020534312, -0.0019444737, 0.030172868, -0.017714826, 0.0023417498, 0.0025127295, 0.01554238, 0.01387952, 0.038916294, -0.004730435, 0.019310636, -0.022770459, 0.055249874, 0.0075566275, -0.023387326, -0.010406287, -0.028617289, 0.025331799, -0.00043289686, -0.028778212, 0.00043708752, -0.02385668, -0.023910321, -0.005706025, -0.0029921427, -0.018170772, 0.023494607, -0.052836046, -0.016011735, -0.0180769, 0.0072414884, -0.0012320587, 0.034061816, -0.02864411, -0.008830593, -0.034169096, 0.011405344, -0.027624937, 0.0018958619, -0.020222528, -0.025170878, 0.010218545, 0.03784348, 0.0024004192, 0.014617079, -0.019471558, -0.014429336, 0.014053851, -0.03269398, 0.0002564694, -0.0150462035, -0.017741647, 0.019404506, -0.006705082, -0.002244526, 0.02377622, -0.013302882, 0.017433213, 0.022797279, -0.0122703, 0.021402622, -0.0029720275, -0.0058166585, -0.01107009, -0.020396858, -0.008662966, 0.018693767, -0.04108874, -0.022649767, 0.003644212, -0.028724572, 0.01555579, 0.0072481935, -0.0023266634, 0.05820011, -0.014496387, -0.019002203, -0.012753066, -0.0067151394, -0.010822002, 0.0007145103, 0.024983136, 0.027517656, 0.0012924045, -0.015394868, -0.040445056, -0.0025429022, -0.011318178, 0.02778586, 0.021080777, 0.0019109483, -0.007107387, -0.0024624413, 0.0081600845, 0.01964589, 0.025144057, -0.028375907, -0.00081718183, 0.033015825, 0.033632692, 0.027450604, -0.003216763, -0.012632375, -0.013074909, -0.019176533, 0.022354743, -0.008763542, -0.007435936, -0.014965743, -0.014040441, 0.011754009, 0.01630676, 0.044521745, 0.0017986383, -0.00815338, -0.005206496, 0.0079790475, -0.016145837, -0.0003668937, 0.00027574648, -0.01813054, 0.006158618, 0.004914825, 0.0035872187, 0.004579571, 0.022582715, -0.030360611, -0.04087418, -0.006104977, -0.01805008, 0.017419802, -0.012417812, 0.01145228, -0.015421689, -0.041598327, -0.025492722, -0.0043616556, -0.0196593, 0.0054881093, 0.0391845, -0.007382295, -0.024969725, -0.019256996, 0.007818125, -0.047150135, 0.032211214, -0.01967271, 0.015850814, -0.01225689, 0.014858462, -0.002492614, 0.026592355, -0.024191935, -0.016025146, 0.017701415, -0.028268626], "content_hash": "2d562d3bc4c833f7a66e92c8b8cef3e8"}, {"content": "the results: | Category | Head | Probability Change | Generation Changed? | |----------|------|-------------------|-------------------| | Previous-Token | | | | | First/Positional | | | | | Bag-of-Words | | | | | Syntactic | | | | | Other | | | | ### Step 4: Analyze Your Results\n\nLook at the table you've filled in:\n- **Which category caused the biggest probability drop?** Previous-Token heads often have the largest impact on simple prompts because local context matters a lot.\n- **Which category had the least effect?** BoW and Other heads often show smaller effects for short prompts.\n- **Did any ablation change the generated text?** A generation change is a stronger signal than just a probability change.\n\n### Step 5: Try a More Complex Prompt\n\nNow repeat the process with a prompt that requires more sophisticated processing:\n\n1. Enter: `The doctors told the patient that they would need`\n2. Analyze and select the generation for comparison.\n3. Ablate one head from each category again and record results.\n\n**What to expect**: For this more complex prompt:\n- **Syntactic heads** may matter more (there are grammatical dependencies like \"doctors...they\")\n- **First/Positional heads** may show more impact because the sentence structure is more complex\n- The pattern of which categories matter may shift compared to the simple prompt\n\n### Step 6: Compare Results Between Prompts\n\n| Category | Simple Prompt Impact | Complex Prompt Impact |\n|----------|---------------------|----------------------|\n| Previous-Token | | |\n| First/Positional | | |\n| Bag-of-Words | | |\n| Syntactic | | |\n| Other | | |\n\n## What You Should Learn", "source_file": "experiment_comparing_heads.md", "chunk_index": 1, "embedding": [0.037110843, 0.006084196, 0.020152178, 0.009325888, -0.011287422, -0.0017980726, 0.022808852, 0.018197527, 0.025713298, 0.06254884, 0.028906813, -0.0026480705, -0.010626695, 0.01309754, 0.005702213, 0.0058673946, -0.00417084, 0.0026773214, -0.020909261, 0.017371617, 0.010227506, -0.011301187, 0.023978889, 0.04247925, 0.034578055, -0.023689821, -0.010702403, 0.036725417, 0.036395054, 0.019477686, 0.00417084, -0.02082667, 0.0007506308, -0.00940848, -0.047572356, -8.435187e-06, -0.017440444, 0.0069169872, 0.026429085, -0.029870372, 0.014081747, -0.018142466, -0.02221695, -0.030723812, -0.014522232, 0.058804717, 0.04046954, -0.06359499, 0.005609298, 0.019216148, -0.031797495, 0.066403076, 0.0064971503, 0.012588229, -0.0046319724, -0.030118145, -0.047682475, -0.010599164, 0.0034516111, -0.028053373, 0.029842842, -0.040579658, 0.03589951, 0.029787783, 0.021982944, -0.0146461185, -0.010255036, -0.003761327, 0.012739645, 0.026002366, 0.05357396, 0.035706796, 0.02179023, 0.009181354, -0.047269523, 0.013125069, 0.02184529, 0.01090888, 0.0008904331, -0.015485792, 0.026236374, -0.020812904, 0.008534392, -0.028053373, -0.014907656, -0.014467171, -0.030641222, -0.02406148, -0.018032346, -0.07686459, -0.03603716, -0.005537031, 0.0064524137, 0.047930248, -0.0060807546, 0.008080143, 0.00017862431, -0.0099384375, -0.011246127, 0.025052572, 0.03422016, -0.037138373, -0.03617481, 0.0078874305, 0.018803194, -0.010950176, -0.0053856145, 0.022244481, -0.037083313, -0.023662291, -0.04782013, 0.006490268, -0.0480679, 0.047434703, 0.01654571, -0.0050758985, -0.04991243, -0.033614494, 0.050463036, -0.04338775, -0.024240427, -0.02039995, -0.029182116, 0.015141664, 0.019161087, -0.0012293998, 0.023689821, -0.042782083, -0.012808471, 0.0018118378, 0.009497953, 0.019188618, 0.0077360137, -0.019945702, -0.051426597, -0.0030765107, -0.0026033337, 0.049637128, -0.026401555, 0.030035555, -0.011122241, -0.021115739, -0.011425073, 0.02868657, -0.07262493, -0.033972386, 0.013971626, 0.03328413, -0.03906549, 0.045810416, 0.023194276, -0.0045252927, -0.043112446, 0.008451802, 0.0074607106, -0.0414331, 0.012037623, 0.012140862, -0.06326462, -0.04812296, 0.00925018, 0.0047902716, -0.024157837, -0.024144072, -0.022093063, -0.008995525, -0.010282566, -0.0039093024, -0.035376433, -0.030008025, -0.039175615, -0.04927923, 0.0032864292, -0.023442049, -0.0031797495, 0.0047765067, -0.053271126, -0.04985737, -0.0076396577, -0.015334376, -0.0014952393, 0.031136766, -0.0050965464, 0.008885404, 0.03157725, -0.022946503, 0.009057468, 0.0072542336, -0.037688978, -0.033972386, 0.024749737, 0.00030648964, 0.088096954, -0.004373876, 0.005836423, -0.051399067, 0.090079136, 0.014591058, 0.012918592, -0.01194815, 0.01949145, -0.011039649, 0.012347339, -0.019395094, 0.019463921, -0.009181354, -0.014714944, -0.0057125366, -0.03686307, -0.016008869, 0.017523035, -0.06695368, 0.04724199, 0.06453102, -0.08049859, -0.003486024, -0.04790272, 0.017068785, -0.022313306, 0.026580503, 0.053133477, 0.0010736816, 0.045507584, -0.018514125, 0.01275341, 0.031164298, 0.051233884, -0.044103537, 0.00112272, -0.045893006, 0.018596716, 0.030861463, -0.010179328, -0.03901043, -0.022519784, -0.017013725, 0.028218556, -0.01088135, 0.019904405, -0.015540853, 0.037771568, 0.0016492369, 0.03592704, 0.003152219, 0.0026446292, 0.028328676, -0.031852555, -0.0314396, 0.02729629, 0.0471594, 0.06563223, -0.042809613, -0.008706457, -0.0046595028, 0.010062324, -0.040139176, -0.01991817, 0.037441205, -0.019078497, 0.0368906, -0.022877678, -0.00043618315, -0.017068785, 0.047765065, -0.011610903, -0.030944055, -0.019078497, -0.020964323, -0.05274805, -0.047352113, -0.0389829, -0.020523837, 0.022478487, -0.004955454, -0.03881772, -0.028466327, -0.01614652, -0.02077161, -0.040029053, 0.041130263, -0.029897904, -0.01160402, 0.009580543, -0.006448972, -0.0010487323, 0.0008912934, 0.011108475, 0.022340836, 0.018651776, -0.03691813, 0.021776466, -0.04360799, 0.034633115, -0.038459826, -0.03061369, -0.015843686, -0.023001565, 0.0037131489, -0.038294643, 0.0680549, 0.041956175, -0.02534164, 0.0021301568, 0.0029749929, 0.031467132, 0.00991779, 0.01983558, -0.020496307, -0.024584556, 0.0016552592, -0.028466327, 0.031467132, -0.004002217, -0.030338388, -0.04157075, -0.011721024, 0.04894887, -0.030916525, 0.00518602, -0.032540813, -0.02951248, 0.010268801, 0.027420176, -0.0069204285, -0.030999115, -0.012429929, -0.018156232, 0.017426679, 0.002689366, -0.007894313, -0.007371237, 0.02868657, 0.03344931, 0.07521278, -0.0060773133, -0.027819365, -0.00065814616, 0.07279011, 0.02269873, 0.025809655, -0.07124841, 0.0070615215, 0.014673648, -0.008761518, 0.007942491, 0.011080945, -0.004800596, 0.019986996, 0.0332566, 0.029980494, -0.025203988, 0.025672004, 0.016380528, -0.011404426, 0.003336328, 0.06739417, -0.02162505, -0.04151569, -0.0033948298, -0.00414331, 0.017344087, 0.014370816, 0.016793482, 0.012663937, 0.018844489, 0.009890259, -0.031852555, -0.0015735286, -0.0030008024, 0.0077841915, -0.02777807, 0.0018600158, -0.0030386567, -0.006376705, -0.07356095, -0.010812525, -0.060401473, 0.00013990984, -0.0115764905, 0.058859777, 0.0036374405, -0.020000761, -0.050490566, -0.003936833, 0.0116797285, 0.036560234, 0.032761056, -0.0013360798, -0.059905928, -0.00026046243, -0.008224676, -0.016944898, -0.0190372, 0.027901957, -0.020592663, 0.069651656, -0.046278432, 0.05919014, 0.051344007, 0.041130263, 0.044103537, 0.018899549, -0.037441205, 0.0074056503, 0.012677702, 0.020386185, -0.00208542, -0.00646962, 0.014260694, -0.0022953385, 0.09674147, 0.027406411, 0.038239583, -0.031522192, 0.028741632, 0.0048694215, 0.023992654, 0.051288944, 0.03697319, 0.027544063, 0.003210721, -0.09206132, -0.0124918725, -0.012925475, -0.023965124, 0.0072335857, 0.014412111, -0.0009850685, 0.020950556, -0.037165903, -0.029154586, 0.053684082, 0.042369127, -0.019615337, 0.010358275, -0.03691813, 0.024667148, 0.044103537, 0.00940848, -0.042837143, -0.0007295529, -0.02162505, 0.026842041, 0.004852215, -0.020248534, 0.017674452, 0.020565132, -0.06254884, 0.020854201, -0.04883875, -0.041680872, -0.0032365306, -0.017578095, 0.010991472, -0.03584445, -0.021473633, 0.041075204, -0.013372842, 0.0662379, -0.022891443, 0.008720222, -0.0017593581, 0.031742435, -0.010206858, 0.012815353, 0.04993996, -0.023703586, -0.013001183, 0.02042748, 0.0002365885, 0.032210447, -0.012946122, -0.018032346, -0.033834737, 0.002911329, 0.006025694, 0.010000381, 0.036697887, -0.011865558, -0.016380528, 0.039946463, 0.0022437193, 0.069871895, 0.037138373, -0.036560234, 0.064310774, 0.027007222, 0.012780941, -0.007811722, -0.0053236713, 0.01932627, -0.0036443232, 0.025795888, -0.015871217, 0.05627193, -0.0011838028, 0.037276022, 0.02034489, 0.021748936, -0.017440444, 0.0027943251, -0.027502768, -0.008637631, 0.08760141, -0.005488853, 0.02692463, 0.06007111, 0.028879283, -0.0015098648, 0.042121354, -0.043002326, -0.024254192, -0.037220962, 0.014687414, -0.0026480705, 0.03446793, -0.013531141, 0.025658237, 0.00044177525, 0.017495504, -0.016669596, 0.02074408, -0.019766754, 0.004435819, -0.033229068, -0.053188536, -0.029429888, -0.017812103, 0.0014289945, -0.020276064, 0.022974033, -0.00855504, 0.0144396415, 0.050352916, -0.03901043, -0.015706034, 0.012842884, -0.01269835, -0.03300883, 0.005069016, -0.0049279234, 0.028356208, -0.011459487, -0.004817802, 0.004662944, 0.0332566, 0.06750429, 0.0021852173, 0.012718998, -0.00039811392, 0.022395898, -0.014453406, 0.019973231, -0.026773214, 0.016793482, -0.01582992, -0.014012922, -0.025520585, -0.0026429086, -0.03314648, 0.031549722, -0.014384581, 0.018830724, -0.0045424993, -0.02093679, 0.027392646, 0.041322976, 0.054867882, 0.0031298508, 0.01529308, 0.02590601, -0.018748133, 0.027736776, 0.03504607, 0.024391843, -0.03300883, -0.006865368, -0.021060677, -0.00288724, -0.0072198207, 0.005437234, -0.037276022, 0.0016113827, 0.03306389, -0.023717351, -0.014412111, 0.032650933, 0.0102343885, -0.004590677, -0.0075020064, -0.007928725, -0.042011235, 0.010489044, 0.040221766, -0.0019271208, 0.022464722, -0.012718998, -0.04881122, -0.04539746, 0.00922265, -0.0036580884, 0.010461513, -0.039478447, 0.029457418, 0.0036615296, 0.023166746, -0.02729629, 0.002882078, 0.010048559, 0.014246929, -0.016408058, -0.012010093, -0.037303552, -0.009325888, 0.0038680069, -0.015279315, 0.005161931, 0.008107672, 0.0032485751, -0.026580503, 0.00075192127, 0.02320804, -0.0077704266, 0.0059499857, 0.016903603, 0.012326691, 0.011218596, -0.039175615, 0.008224676, 0.0043050502, -0.00925018, 0.017798338, -0.008947346, 0.010268801, -0.017357852, 0.0202623, 0.026552971, 0.031219358, -0.0030937172, -0.021803996, -0.0033810646, 0.008623865, -0.019477686, -0.028466327, -0.045893006, 6.183563e-05, -0.008210911, 0.02122586, -0.028025843, -0.023194276, -0.013331546, -0.014687414, 0.05737314, -0.0058054514, 0.023538405, -0.024309253, 0.004601001, -0.0065384456, -0.022877678, -0.009738843, -0.0045080865, -0.010062324, -0.03344931, -0.028328676, 0.016697126, 0.030283328, -0.020523837, 0.045837946, -0.002823576, 0.018431535, -0.08148968, 0.012161509, 0.039918933, 0.020276064, -0.023056624, 0.032623403, -0.009277711, -0.03336672, -0.006166787, 0.0037028252, -0.019574042, 0.017688217, -0.04369058, -0.03050357, 0.011851793, -0.010199975, 0.009876494, -0.02133598, 0.0031504985, 0.032650933, -0.021005617, -0.03865254, 0.022258246, -0.004683592, -0.004253431, -0.009043703, 0.0046801507, 0.044516493, -0.000267345, -0.06012617, -0.020248534, 0.0072886464, -0.015609679, -0.01790846, -0.0009919511, -0.018348943, -0.021294685, 0.041020144, 0.024598321, -0.012333574, -0.007901195, 0.0010874468, -0.013503611, 0.0039127436, 0.0007403069, 0.008995525, -0.013372842, -0.027489003, 2.3094264e-05, -0.051399067, -0.0130080655, -0.005664359, -0.014047335, -0.0020286387, -0.013221426, -0.018541656, 0.01790846, 0.010358275, -0.024006419, 0.017757041, 0.01938133, 0.010674873, 0.014398346, -0.040276825, -0.019023435, 0.0076603056, -0.03336672, 0.03138454, 0.021528693, -0.00994532, 0.024020186, -0.028328676, -0.015072838, 0.005943103, 0.003534202, -0.01585745, -0.021046912, 0.043084916, 0.0020527279, -0.046718914, -0.0009377508, 0.012684585, -0.005870836, -0.033614494, 0.060621716, 0.003785416, 0.008114555, -0.03975375, -0.014907656, 0.045810416, 0.019367564, 0.019945702, -0.015595914, 0.0096769, -0.01205827, -0.028108435, 0.014260694, -0.009401597, -0.0341651, 0.0040194234, 0.024171602, -0.005299582, 0.03504607, -0.011383778, 0.0556112, -0.033834737, 0.025768358, -0.00028412128, -0.01090888, 0.004766183, -0.0011751996, 0.013916565, -0.0719642, -0.007247351, 0.0023744882, 0.020950556, 0.009174472, -0.029044464, -0.0017705422, -0.02775054, -0.013283369, -0.030090615, 0.021514928, -0.003668412, -0.012402399, -0.014123043, -0.016284171, -0.10180704, 0.013778914, 0.037276022, -0.03884525, -0.029622601, -0.021556223, 0.0281222, 0.0073643546, -0.04341528, 0.004298168, 0.018114936, -0.0057538324, 0.0044289366, -0.00625626, 0.010915763, -0.031962674, 0.01154896, -0.043084916, 0.020468777, -0.056712415, -0.021446101, -0.011438839, -0.026043661, 3.6617446e-05, -0.03402745, -0.004800596, -0.025630707, -0.0032038384, -0.004184605, -0.0057469495, -0.027392646, 0.010716168, -0.006125491, -0.007667188, -0.011899971, -0.0075639496, -0.0150590725, -0.0026463498, -0.018335178, -0.026277669, 0.025382934, -0.015031543, 0.0015391157, -0.013586202, 0.018940846, -0.026621798, 0.0048315674, -0.005203226, 0.021115739, 0.013359077, 0.017233966, 0.02534164, 0.0041983705, -0.01248499, 0.0062321713, -0.004473673, 0.0039712456, 0.0069823717, 0.0071922904, 0.045204747, 0.021170799, 0.010000381, 0.011445722, -0.02553435, 0.007894313, -0.019973231, -0.016284171, 0.01801858, -0.021638814, -0.008665161, -0.02686957, 0.017467974, -0.012718998, 0.008789048, 0.0047283284, 0.008444919, 0.016793482, -0.00791496, -0.022395898, 0.03141207, 0.070862986, -0.030751342, 0.0031487779, 0.049334295, -0.006025694, 0.015540853, 0.020523837, -0.00015378253, -0.0037441205, 0.035706796, 0.015747331, -0.0317149, -0.012966771, 0.018459065, -0.021859057, -0.012608876, 0.023676056, 0.036780477, 0.013861505, 0.0036546472, -0.0006886876, 0.029264707, -0.010344509, -0.021459866, -0.015100368, 0.029760253, -0.0023882533, 0.021170799, -0.01379268, 0.0009239856, 0.02817726, 0.015761096, 0.008789048, 0.046773978, 0.0015408364, 0.01759186, 0.04784766, -0.020055821, 0.01761939, -0.018720603, 0.021748936, 0.035514083, -0.0057572736, 0.0026188195, -0.04162581, 0.03314648, -0.029319767, 0.011831146, 0.017467974, -0.0098351985, 0.03135701, 0.00097646523, -0.027571592, -0.014618589, -0.0048315674, 0.001718923, 0.009725078, -0.02023477, -0.013076891, 0.0060394593, -0.022010474, 0.014274459, 0.032540813, -0.014549762, -0.014178104, -0.040552128, 0.0020630518, -0.016463118, -0.02809467, -0.012354221, 0.0076121273, -0.009380949, -0.013304017, 0.009553013, -0.014012922, 0.0023159862, 0.031797495, 0.0010384084, -0.010812525, 0.0037097076, -0.0034688176, 0.01852789, 0.042286538, -0.00062803493, 0.01205827, -0.039891403, -0.01248499, -0.003606469, 0.0081971465, 0.0043325806, -0.048398264, 0.002288456, 0.003210721, 0.0062528187, 0.016834777, 0.03240316, 0.005089664, 0.007116582, -0.012250982, 0.0122441, -0.029925434, 0.043029856, -0.01708255, -0.013331546, 0.017206436, -0.004976101, 0.00689634, 0.021088209, 0.0048694215, -0.028411267, -0.001291343, 0.033807207, 0.0049451296, 0.054730233, -0.024763502, 0.0043876413, 0.025672004, -0.012478108, 0.0062252884, -0.025727063, -0.068660565, 0.020867966, -0.005970633, 0.0008370931, 0.014150573, 0.037165903, -0.000108077926, -0.0041398685, -3.6886297e-05, 0.026773214, -0.028218556, 0.01790846, -0.014921421, 0.004573471, -0.028576449, -0.014825066, -0.011707259, 0.005619622, 0.0004654341, -0.03611975, -0.05090352, -0.010365157, -0.027791835, 0.030008025, 0.0055645616, 0.017770808, 0.0268971, -0.00895423, 0.01611899, -0.0019271208, -0.024336783, -0.030751342, 0.0024450347, 0.002476006, -0.01269835, 0.020110883, -0.051894613, -0.008871638, -0.027709246, 0.011597138, 0.02724123, 0.0031160857, 0.005031162, 0.0025310668, -0.0130080655, -0.0317149, 0.021872822, -0.0007033131, -0.0025964512, -0.0043566697, -0.017784573, 0.008740869, 0.04154322, -0.04242419, 0.026828276, -0.021487398, 0.017674452, 0.028851751, 0.015114133, 0.01790846, -0.006614154, 0.005402821, 0.0015193283, -0.040579658, -0.005492294, -0.0096769, -0.0055439137, -0.0057331845, 0.027599124, -0.03972622, -0.013407255, 0.033889797, 0.0014264135, 0.036725417, -0.0071716425, -0.02585095, -0.0030593043, -0.009188237, 0.006734599, -0.007522654, 0.018706838, 0.027544063, -0.0064833853, 0.008327915, 0.005247963, -0.016435588, -0.00580201, -0.022547314, 0.00051748357, 0.005065575, 0.018651776, 0.00087365677, -0.009828316, -0.0013145717, -0.012808471, -0.020881731, 0.025217753, 0.024474435, 0.020647723, -0.0317149, -0.0038955372, 0.011638434, 0.054895412, -0.050160203, -0.0019512099, 0.032623403, 0.04322257, -0.039203145, 0.014074865, -0.0040916903, -0.020510072, 0.016490648, 0.0007854738, 0.007894313, -0.020592663, -0.01790846, -0.0129048275, -0.014033569, 0.0037406792, -0.016862307, -0.0072542336, 0.011039649, 0.015375671, 0.014673648, 0.008438036, -0.020785375, -0.0015563221, 0.01651818, 0.022478487, 0.0012319809, 0.024171602, 0.007976904, -0.021473633, -0.017165141, -0.011480134, 0.027736776, -0.03232057, 0.007343707, 0.01799105, -0.016105223, -0.02545176, 0.017454209, -0.013572437, -0.017605625, 0.03884525, -0.00946354, -0.0051378417, -0.008066378, 0.015733566, 0.021446101, 0.08330668, -0.039616097, -0.018885784, 0.009477305, 0.012113331, 0.015499557, -0.014480936, -0.0068137487, -0.058474354, -0.012712115, -0.01949145, 0.03686307, 0.0044117304, 0.0058914837, 0.024267958, 0.0014074865, -0.016022634, 0.0089060515, 0.06695368, -0.0063801464, 0.022409663, 0.044048477, -0.023331927, 0.009146942, -0.010901998, 0.03328413, 0.008603218, -0.0044667907, 0.004394524, 0.031494662, 0.014976482, -0.030888993, -0.010688638, -0.027764305, -0.045893006, 0.015554618, -0.0009824875, 0.018624246, -0.010936411, 0.013758266, -0.034660645, 0.013269603, 0.021459866, -0.021046912, 0.016807247, 0.015747331, 0.009057468, 0.0049623363, 0.022767557, -0.018541656, -0.035734326, 0.00021948961, -0.031522192, -0.0317149, 0.04713187, -0.060291354, -0.0067139515, 0.0074951234, -0.010303214, 0.0066588908, -0.022065533, 0.04165334, -0.023111684, -0.017646922, -0.0067621293, -0.046278432, -0.02553435, 0.006569417, -0.029209645, 0.007240468, -0.04355293, -0.0024364314, 0.010805642, -0.0050758985, 0.00994532, -0.007150995, 0.007102817, -0.023345692, 0.035486553, -0.005726302, 0.012629524, -0.03328413, -0.05239016, -0.024956215, -0.001446201, -0.0011992886, 0.04498451, -0.0025293462, -0.028493859, 0.0024071804, -0.046498675, 0.018638011, 0.0368906, 0.062493775, 0.00510687, 0.002476006, 0.020620193, 0.010915763, -0.0041742814, -0.040910024, 0.0022678082, 0.0032176035, -0.0066554495, 3.470861e-05, -0.027819365, -0.029980494, -0.048370734, 0.0010341068, 0.009291476, 0.002360723, 0.032926235, -0.0016036398, 0.014742474, -0.005664359, -0.016848542, -0.0073230593, -0.022561079, 0.025961071, -0.0059947222, -0.05373914, 0.0056024157, 0.0036099104, -0.005956868, 0.013751384, 0.003940274, 0.008438036, 0.012037623, -0.004095132, -0.011617785, 0.026552971, 0.0012061711, 0.016476883, 0.017825868, 0.030778874, 0.018858254, -0.026057428, -0.007116582, -0.022327071, 0.0019752989, 0.006855044, 0.00071320677, 0.027461473, -0.034605585, -0.022946503, -0.0016604211, 0.006352616, -0.008747752, -0.0031917938, 0.01657324, 0.0017404311, -0.02959507, -0.019202383, 0.016848542, 0.0064386483, -0.00940848, -0.004287844, -0.014150573, -0.020991853, 0.0020097117, -0.020895496, 0.022684965, -0.009959085, 0.002969831, -0.008912934, -0.0013489845, 0.011741672, -0.012987418, -0.0002849816, 0.019092262, -0.008713339, 0.00027444266, -0.027695479, -0.01673842, 0.010647343, -0.0073092943, -0.00077816105, -0.0040194234, 0.041020144, 0.0042121354, 0.03245822, -0.014384581, -0.018087406, -0.008830343, -0.006011929, -0.014191869, 0.027516533, 0.011955032, -0.004955454, -0.004879745, -0.0051584896, -0.024983745, -0.03419263, -0.01951898, 0.0010341068, 0.027406411, 0.039203145, -0.037248492, -0.0087684, 0.025493056, 0.0064042355, 0.0037716508, -0.021542458, -0.011191066, -0.015995104, -0.01898214, 0.024694677, 0.0012397238, -0.0278056, 0.003630558, 0.0055439137, 0.0072886464, -0.002240278, 0.004934806, -0.012519403, 0.026566738, -0.021693874, 0.021528693, -0.006627919, -0.019023435, -0.0113700135, -0.03408251, -0.00505181, 0.034522995, 0.0057710386, 0.00077816105, 0.009243297, 0.0028356207, -0.015623444, -0.023152981, 0.00473177, 0.018858254, -0.023731116, -0.01580239, 0.01708255, 0.0058914837, 0.01475624, -0.046966687, 0.012650172, 0.010743699, 0.01585745, -0.02301533, 0.028135965, 0.003307077, 0.019560277, -0.033724613, 0.060511597, 0.005065575, 0.01796352, -0.0024415932, -0.004662944, 0.041735932, -0.0073024114, -0.023428284, -0.010358275, -0.026910866, -0.054069504, 0.0066416846, -0.014494702, -0.024157837, 0.026071193, -0.04647114, -0.039175615, -0.0086101005, -0.0021508045, 0.006011929, 0.023689821, -0.0097319605, -0.053050883, -0.038432296, 0.03243069, 0.0047352114, 0.015403202, -0.010138032, -0.022932738, -0.00082203746, 0.028328676, 0.027089814, -0.02352464, -0.0032657816, -0.0016027795, -0.00836921, -0.020014526, -0.015540853, 0.009821434, -0.012705232, 0.035761856, -0.014067982, 0.01801858, 0.016669596, -0.037771568, 0.025121396, 0.028769162, 0.0017361294, 0.0133177815, 0.003737238, 0.013544907, 0.023359457, -0.016421823, -0.024157837, 0.0057297433, -0.042093825, 0.007949374, 0.005130959, -0.029044464, -0.0010538942, 0.029842842, -0.023882532, 0.02179023, -0.017798338, 0.0061289323, -0.0024364314, -0.014246929, 0.012829118, 0.011452604, 0.0042499895, 0.041763462, -0.00743318, -0.007908078, -0.012381751, 0.023593465, -0.028576449, 0.01796352, 0.0020355214, 0.009029938, 0.011665964, -0.003104041, -0.0045941183, 0.003544526, -0.028025843, -0.030173207, 0.007970021, 0.03804687, 0.024075245, -0.016063929, 0.016600769, -0.022588609, -0.025011275, -0.014095512, 0.02681451, -0.020014526, -0.0029319767, -0.01475624, -0.0030954378, -0.015086603, -0.00034068743, 0.039285734, -0.012601994, -0.0006422302, 0.0026136576, 0.0061874343, -0.0057366258, -0.010303214, 0.0101449145, -0.0032382512, 0.01790846, -0.011838028, 0.02553435, 0.026429085, -0.0028906814, -0.013538024, -0.0317149, -0.029402358, -0.0130080655, 0.04421366, -0.005402821, 0.027516533, -0.004587236, -0.045782886, -0.017688217, 0.004494321, -0.01989064, 0.029292237, 0.022629905, -0.016958663, -0.011955032, -0.010034794, -0.0069789304, -0.02959507, 0.021404807, -0.057868686, 0.034770764, -0.0014401787, 0.009394714, 0.011968797, 0.035624206, -0.016724655, -0.027530298, -0.00092140463, -0.0062252884], "content_hash": "43ac36dd32554e694ad9b0255a3ba4c6"}, {"content": "prompt ### Step 6: Compare Results Between Prompts | Category | Simple Prompt Impact | Complex Prompt Impact | |----------|---------------------|----------------------| | Previous-Token | | | | First/Positional | | | | Bag-of-Words | | | | Syntactic | | | | Other | | | ## What You Should Learn\n\n- No single head category is always the \"most important\" -- it depends on the prompt\n- Simple prompts tend to rely more on Previous-Token heads (local patterns)\n- Complex prompts with grammatical dependencies may rely more on Syntactic heads\n- Some heads are redundant for certain inputs but critical for others\n- Ablation is most informative when you compare across conditions (categories, prompts, or both)\n\n## Advanced Challenge\n\nTry ablating **two heads simultaneously** from the same category. Does removing two Previous-Token heads have a bigger effect than removing one? Or does the model have enough redundancy to compensate?", "source_file": "experiment_comparing_heads.md", "chunk_index": 2, "embedding": [0.032853976, 0.00626801, 0.026533764, 0.012689153, 0.018974563, 0.004600949, 0.03808834, 0.024278533, 0.024529114, 0.050311133, 0.03274261, -0.02425069, -0.021020977, -0.0022743721, -0.012793561, -0.00697799, -0.0046601137, 0.002331797, 0.005961744, 0.06921609, 0.011937409, -0.016719334, 0.024932828, 0.044464238, 0.038338922, -0.015689166, -0.0015234988, 0.0340512, 0.035415474, 0.014798211, 0.0054919044, -0.020714711, -0.01621817, -0.00579469, -0.034608044, 0.029763475, 0.006591677, 0.02401403, 0.018264584, 0.005039466, 0.018194977, -0.014714683, -0.039313402, -0.01195133, -0.019225145, 0.040371414, 0.044659134, -0.03939693, -0.01585622, 0.032798294, -0.04362897, 0.050255448, 0.017652052, 0.011582419, 0.0034976956, -0.010433922, -0.03831108, -0.018737905, 0.013670595, -0.014178718, 0.0012790081, -0.04555009, 0.029791318, 0.017234417, 0.014136955, -0.045494407, -0.002850361, -0.010503528, 0.0064176624, 0.01953141, 0.02934584, 0.05122993, 0.03822755, 0.007475672, -0.020547656, 0.008067322, 0.020840002, 0.01571701, 0.042654485, -0.025893388, -0.008965238, 0.004200715, 0.026241418, -0.03783776, -0.022858573, -0.020408446, -0.03961967, -0.017777342, -0.028677624, -0.060974754, -0.01490958, 0.005704202, 0.01419264, 0.021160189, 0.0019298231, -0.0034976956, 0.008853869, -0.0083457455, -0.02200938, 0.052677732, 0.0134965805, -0.043963075, -0.036724064, 0.01773558, 0.015062713, -0.024306376, 0.017958319, 0.0089304345, -0.009598651, -0.029178787, -0.020310996, 0.012626507, -0.033550035, 0.04507677, 0.019475726, -0.025921231, -0.050311133, -0.009368952, 0.029791318, -0.07071958, 0.010440882, -0.031851653, -0.02745256, -0.008700736, 0.0111160595, -0.010106774, 0.029485052, -0.046301834, -0.021466454, -0.024292454, 0.01644091, 0.018181058, 0.0010327772, -0.0012850986, -0.034942154, -0.007239012, -0.032464184, 0.04841785, -0.040176515, 0.04493756, 0.006494229, -0.020436287, 0.00017042566, 0.03430178, -0.053178895, -0.02352679, 0.005070789, 0.03218576, -0.026157891, 0.035081364, -0.0104826465, -0.010197262, -0.034747258, -0.025475753, -0.024640484, -0.03769855, -0.006281931, -0.0019611458, -0.043239176, -0.049865656, 0.03160107, -0.02638063, -0.0165662, -0.008860829, -0.031211277, -0.0011537175, -0.034969997, -0.00043938713, -0.047749635, -0.03513705, -0.02494675, -0.0356939, -0.026589448, -0.06548522, 0.0011867803, -0.0007634895, -0.04886333, -0.042988595, -0.027703142, -0.009452479, -0.00068561797, 0.027021004, 0.017958319, 0.041457266, 0.02283073, -0.016524436, -0.006581236, 0.022301726, -0.061086122, 6.737632e-05, 0.025239093, 0.029485052, 0.08742499, 0.018946722, 0.006553394, -0.04864059, 0.05084014, 0.030376008, 6.8300775e-05, -0.0089304345, -0.0038666069, -0.005095151, 0.0029199668, -0.020742554, 0.022914257, -0.01313463, -0.021188032, -0.0069257854, -0.053652216, -0.03522058, 0.008255258, -0.058357574, 0.012250636, 0.05086798, -0.06754555, -0.022969943, -0.048919015, 0.0021473416, -0.010670582, 0.010343434, 0.06348056, -0.0014373615, 0.063703306, -0.012863168, -0.016594043, 0.042515274, 0.041930586, -0.042431746, -0.009591691, -0.03939693, 0.03987025, 0.021257637, -0.017164811, 0.0034750737, -0.047638267, -0.016747175, 0.012988458, -0.031155594, 0.014234403, -0.03143402, 0.018765746, -0.0004139375, 0.041568633, -0.008387509, -0.0064281034, 0.016399147, -0.026464157, -0.029986214, -0.0031200838, 0.040983945, 0.07634373, -0.01644091, -0.03808834, 0.012132306, -0.00851976, -0.017206574, 0.007872425, 0.046218306, -0.046942208, 0.01987944, -0.024988512, 0.028065093, 0.009842272, 0.0544318, -0.036000162, -0.043684654, -0.017652052, -0.004520902, -0.037726387, -0.042515274, -0.049837813, -0.033661406, 0.03597232, -0.019322593, -0.023582473, -0.028566254, -0.019183382, -0.026909634, -0.025782019, -0.00721813, -0.022329567, -0.016051117, 0.01915554, -0.0040267003, -0.012153188, -0.005157796, 0.026074365, 0.041429423, 0.02756393, -0.027410798, -0.007997716, -0.058580313, 0.032798294, -0.04627399, -0.0071485243, -0.014199601, -0.0075313565, 0.0077053714, -0.008436233, 0.03087717, 0.038338922, -0.00532833, -0.005665919, -0.0018410756, 0.044380713, -0.01076107, 0.003323681, 0.0017705996, -0.02945721, -0.009807469, -0.028357437, -0.010197262, -0.011860843, -0.010468725, -0.023707764, -0.034468833, 0.0153829, -0.02331797, 0.023666, -0.013872453, -0.034273937, 0.004315565, 0.026686896, 0.007503514, 0.006964069, 0.009793548, -0.033633564, 0.008109085, -0.014798211, -0.0007782808, -0.03535979, -0.005015104, 0.018946722, 0.059359897, -0.032130077, -0.024529114, -0.004994222, 0.04671947, 0.0024518673, 0.050171923, -0.05654782, 0.0032801772, 0.012891009, 0.018529085, -0.0134478565, 0.050338976, 0.0057668476, 0.01857085, 0.032937504, 0.027870195, -0.021870168, 0.0071833273, 0.013733241, -0.00037282653, 0.0041589513, 0.05418122, -0.041986268, -0.05075661, -0.009069647, 0.011937409, 0.03157323, 0.0036473484, 8.34183e-05, 0.016162487, 0.012034858, -0.0066682436, -0.010670582, -0.017958319, -0.015536034, 0.0067726523, -0.014505866, 0.013475699, -0.0147842895, -0.028106855, -0.066877335, 0.010935084, -0.07394929, -0.013942058, 0.011902606, 0.036612693, 0.0045452644, -0.062422555, -0.043963075, -0.0053666136, 0.02046413, 0.027814511, 0.04708142, -0.012215832, -0.071721904, -0.031851653, 0.006946667, -0.016580122, 0.009355031, 0.020394525, -0.029123101, 0.062032763, -0.025962995, 0.06615343, 0.033661406, 0.059694007, 0.024069715, 0.026784344, -0.051870305, 0.008944356, 0.030014057, 0.014352733, 0.03143402, -0.010252947, 0.008798184, 0.012494256, 0.08035303, 0.010489606, 0.02887252, -0.00402322, 0.0049246163, -0.012487296, -0.004764523, 0.023624238, 0.029568579, 0.020324918, 0.024515193, -0.067712605, -0.023457183, -0.04284938, -0.015828378, -0.012424651, 0.03513705, -0.010440882, 0.01822282, -0.02768922, -0.024306376, 0.0307658, 0.06993999, 0.0071485243, 0.011317917, -0.022663675, 0.007468711, 0.06420447, 0.0066195196, -0.03491431, 0.023638159, -0.02459872, 0.018000081, -0.006803975, -0.044185814, 0.023331892, 0.027062768, -0.057410933, 0.01598151, -0.017471077, -0.033410825, -0.004423454, -0.019698465, 0.03243634, -0.012570823, -0.015368979, 0.031378333, -0.00709632, 0.09026491, 0.0029112662, 0.010837636, -0.011373602, 0.03196302, -0.021076662, 0.034802943, 0.077624485, -0.01313463, -0.010949005, 0.032714766, 0.001218103, 0.02507204, -0.023930503, 0.00040306157, -0.026297104, -0.0074478295, -0.014603314, 0.0036160257, 0.02685395, 0.00030648342, -0.038366765, 0.017192652, -0.023596395, 0.053401634, 0.013273842, -0.03728091, 0.077067636, 0.035721738, 0.0011989614, -0.025879467, -0.00010381065, 0.01621817, -0.003925772, 0.027758826, -0.01609288, 0.03174028, 0.002352679, 0.034246095, 0.03574958, 0.046051253, -0.016037196, 0.007823701, -0.004357328, -0.0008783392, 0.07701195, 0.0009214079, 0.019726308, 0.06710007, 0.0076636076, -0.0062401677, 0.037336595, -0.05415338, -0.012194951, -0.01632954, 0.025754178, -0.0125499405, 0.033661406, -0.024640484, 0.033550035, 0.016705412, 0.021675272, 0.002424025, 0.01749892, -0.03207439, -0.011575459, -0.014164797, -0.040399253, -0.011791237, -0.043601125, -0.027577851, 0.0021577824, 0.0054362197, -0.02898389, 0.008770341, 0.036835432, -0.024556955, 0.0024936309, 0.0066055986, -0.0011537175, -0.031628914, -0.004413013, 0.021327242, 0.0072946968, -0.007879386, -0.015522112, 0.009208858, 0.023011705, 0.06910472, -0.008728578, 0.017958319, 0.018250663, -0.00061296683, -0.007628805, 0.017345786, -0.025517518, 0.002801637, -0.019893361, -0.02035276, -0.018723983, 0.0022621911, -0.02567065, 0.009055725, -0.03655701, 0.023401499, -0.017944397, -0.013677556, 0.036389954, 0.04092826, 0.040510625, 0.007503514, -0.004687956, 0.014394497, -0.009076607, 0.02873331, -0.009410716, 0.015786614, -0.014095192, -0.0034211292, -0.008819065, 0.013120709, -0.021842327, -0.0005859945, -0.043127805, 0.0017775602, 0.03499784, -0.022259962, -0.011763395, 0.048222955, 0.003348043, -0.026352787, 0.013858532, -0.014895659, -0.044074446, 0.024932828, 0.03430178, -0.024181085, 0.026199656, 0.0034733335, -0.035443317, -0.029373683, 0.0054362197, 0.0030017537, -0.0044860994, -0.012466414, 0.023039548, -0.0013425235, 0.021814484, -0.022162514, 0.02271936, 0.005220441, 0.018835353, 0.021132346, -0.010684503, -0.022552306, -0.0074965535, -0.0067865737, -0.00063776394, 0.009424637, 0.008686814, 0.0014591133, -0.039118506, -0.0037935206, 0.013378251, -0.019712387, 0.010440882, 0.03452452, 0.017345786, 0.013830689, -0.04944802, 0.022385253, -0.0044164932, -0.020881765, 0.02128548, 0.0038666069, 0.008721617, -0.0033393423, 0.010496567, 0.008568484, 0.034023356, 0.005098631, -0.019962968, -0.016942073, 0.01076107, -0.023749528, -0.05039466, -0.044269342, -0.010378237, 0.0077332137, 0.027410798, -0.033772774, -0.012828364, -0.0033306417, -0.03455236, 0.057076823, -0.0028729828, -0.002377041, -0.015508191, 0.0006769172, 0.02306739, -0.0418749, 0.014032546, -0.0037413163, -0.018014003, -0.03146186, -0.032826133, 0.0012790081, 0.0058364533, -0.005380535, 0.041234527, -0.01241769, -0.0027476922, -0.056102343, 0.013635793, 0.072668545, 0.046385363, 0.00793507, 0.022232119, 0.0035290183, -0.03204655, -0.028092934, -0.007155485, -0.03800481, 0.023011705, -0.046441045, -0.040427096, -0.0007465231, -0.013176394, -0.0035812226, -0.01787479, -0.0018045325, 0.03204655, -0.018793589, -0.04889117, 0.0340512, -0.0133504085, 0.0076218443, -0.014338812, 0.014185679, 0.044798348, -0.020018652, -0.016844625, -0.027675299, 0.0010710605, -0.027021004, -0.007510475, 0.007026714, -0.026631212, 0.013426975, 0.053290267, 0.012939733, -0.026533764, -0.030376008, -0.0025823782, -0.0051055914, 0.012111424, 0.0076844897, 0.0061357585, -0.025601044, -0.033772774, 0.00043242655, -0.03455236, -0.02614397, -0.02093745, 0.0010040649, 0.030821485, 0.026046522, 0.0073155784, -0.003041777, 0.0044791386, -0.0053909756, 0.01432489, 0.03705817, 0.011930449, -0.004980301, -0.018139293, 0.00032344984, 0.0010458285, -0.022023302, 0.029262314, 0.01432489, -0.0023474584, 0.042320378, -0.026575526, -0.005157796, 0.003099202, 0.027772747, -0.011095178, -0.020046495, 0.046552416, 0.0045383037, -0.03251987, -0.022037223, 0.013266882, -0.0043851705, -0.01811145, 0.043378387, 0.009487282, -0.018194977, -0.016190328, -0.013907256, 0.049726445, 0.010148538, 0.0072807753, -0.009668257, -0.0035673014, -0.013259921, -0.0130163, -0.008617208, -0.023220522, -0.02707669, 0.0050429464, 0.03391199, -0.015396821, 0.031378333, -0.036278587, 0.04635752, -0.03243634, 0.028176462, -0.004781924, -0.03274261, 0.021215873, 0.012689153, 0.019002406, -0.05393064, -0.016385226, 0.009104449, 0.010900281, -0.018501244, -0.025545359, 0.0031479262, -0.033772774, -0.020979213, -0.016148565, -0.005359653, 0.0046113897, -0.009215819, -0.003821363, -0.027842354, -0.13965724, 0.0023161357, 0.03800481, -0.033856302, -0.045355193, 0.018083608, 0.027494324, 8.151501e-05, -0.027118452, 0.011672907, 0.0036995527, 0.001348614, -0.02306739, -0.026199656, 0.03391199, -0.033382982, 0.010183341, -0.03505352, 0.0059791454, -0.056102343, -0.017095204, 0.0013442637, -0.0021943254, -0.0063967807, -0.049949184, -0.011937409, -0.038283236, -0.018083608, -0.00971002, 0.018654376, -0.02756393, -0.0077888984, -0.023443261, 0.00038152726, -0.0151740825, 0.02024139, -0.013907256, -0.026297104, -0.00026080455, -0.013559226, 0.023401499, -0.018125372, -0.020380603, -0.019809835, 0.020297077, -0.035443317, 0.0035411993, -0.024863223, 0.031183437, 0.020380603, 0.009487282, 0.028371358, -0.0054501407, 0.005004663, 0.010301671, -0.018236741, 0.004082385, -0.007005832, 0.027842354, 0.044269342, 0.019294752, -0.016983835, 0.0096821785, -5.394456e-05, 0.0022813329, 0.0030226356, -0.018543007, 0.038144026, -0.0072738146, -0.012480335, -0.014450181, 0.014157836, -0.00922974, 0.0011406664, 0.006407222, 0.005568471, -0.012932774, -0.003182729, -0.037949126, 0.024988512, 0.08063146, -0.031044224, -0.005046427, 0.030014057, 0.008464076, 0.033522192, -0.0005346602, -0.01833419, -0.0025997797, 0.03243634, 0.013510502, -0.02792588, -0.015494269, 0.042181164, -0.046775155, 0.012020936, 0.028816836, 0.025503596, -0.003535979, 0.027243743, 0.0378656, 0.015508191, -0.0033410825, -0.025809862, -0.027730985, 0.0015565616, -0.0071485243, 0.019949047, 0.0088817105, 0.013232078, 0.03335514, 0.015355058, 0.0013277322, 0.055879604, 0.005394456, -0.0010754109, 0.041819215, -0.005819052, 0.025141645, -0.0145893935, 0.008317903, 0.029791318, -0.015396821, 0.012000054, -0.025782019, 0.04841785, -0.018014003, 0.015090556, 0.007064997, 0.015285452, 0.04318349, 0.0094664, -0.026631212, -0.022427015, 0.0056241555, 0.0021351604, 0.00077175524, -0.01845948, -0.005752926, 0.0016218171, -0.013635793, 0.027021004, 0.047220632, -0.015577797, 0.0051334337, -0.028538411, 0.01100469, -0.0111160595, -0.031183437, -0.009125331, 0.014575472, 0.0041554715, -0.013614911, 0.009647376, -0.013071985, -0.0015757032, 0.026672974, -0.012891009, 0.00082395965, 0.032241445, -0.028788993, 0.021870168, 0.04627399, -0.014185679, 0.016621886, -0.029011732, -0.007990755, -0.001762769, 0.0031444458, 0.012612586, -0.04329486, -0.025962995, 0.006341096, 0.0070754383, 0.02128548, 0.039118506, -0.026297104, 0.0024588278, 0.0024449066, 0.009334149, -0.030376008, 0.04084473, -0.023345813, -0.011860843, 0.021856248, 0.029512895, 0.009007001, 0.017373629, -0.004165912, -0.014951344, 0.023150917, 0.014352733, 0.0011493672, 0.036974646, -0.02104882, 0.014561551, 0.024041872, 0.0021508217, 0.014714683, -0.019211223, -0.03775423, 0.018069688, 0.003821363, -0.021591745, 0.0027790149, 0.03505352, -0.0013825468, -0.02601868, 0.006222766, 0.0050359857, -0.030570904, 0.03277045, -0.026742581, 0.0032262327, -0.03204655, 0.018793589, -0.0032610358, -0.0119652515, 0.008116046, 0.0022291285, -0.054932963, -0.0016705412, -0.020951372, 0.006720448, 0.029206628, -0.006542953, 0.036863275, -0.020728633, 0.028482728, 0.009320227, -0.02923447, -0.016594043, -0.014728605, 0.011728591, -0.028106855, 0.001620077, -0.062422555, 0.002105578, -0.011812119, 0.015327215, 0.035081364, 0.0069362265, -0.011512813, -0.0012703073, -0.00061818724, -0.038478132, 0.006929266, 0.016120723, -0.011561537, 0.0030957216, -0.010705385, -0.008554563, 0.040816892, -0.03900714, 0.031545386, -0.01587014, 0.02472401, 0.01809753, 0.023958346, 0.0102668675, -0.004127629, 0.013232078, 0.00130076, -0.036807593, -0.021800563, -0.0073573417, -0.006814416, -0.0047958456, 0.0004306864, -0.04788885, -0.00354816, 0.017916555, -0.010468725, 0.039229877, -0.016761096, -0.023457183, 0.0120139755, 9.937871e-06, -0.015591718, -0.036055848, 0.02710453, 0.023011705, 0.0020481532, 0.00957777, -0.0042389985, -0.024807537, -0.0026363228, -0.027884116, 0.0051334337, -0.009222779, 0.021563902, -0.0067239283, 0.0040997867, 0.012160148, 0.0040162597, -0.023373656, 0.018877115, 0.010607936, 0.020617263, -0.05454317, 0.0067343693, 0.006494229, 0.037420124, -0.03705817, -0.015536034, 0.027953723, 0.022065066, -0.03290966, 0.020756474, -0.013809808, -0.016552279, 0.013329526, -0.007071958, 0.031406175, -0.01574485, -0.007976835, -0.0052343626, -0.030849328, 0.016635805, -0.033160243, -0.010085893, 0.0316846, 0.0064176624, -0.003964055, 0.0077332137, -0.02898389, -0.00039240316, 0.0022152073, 0.0036995527, -0.002199546, -0.002091657, 0.0011284854, -0.0041798335, 0.003501176, -0.021549981, 0.03925772, -0.036250744, 0.008777302, -0.001954185, -0.024278533, -0.03321593, 0.022496622, 0.025280857, -0.020533735, 0.030710116, 0.01147105, -0.010990769, -0.0045452644, 0.009396794, 0.018891037, 0.08057577, -0.04070552, -0.027160216, 0.010447843, -0.00922974, 0.011262232, -0.020199627, -0.011436246, -0.035777424, -0.010044129, -0.036863275, 0.061977077, 0.00013638403, 0.00745479, -0.008290061, 0.010559212, -0.0098283505, 0.03421825, 0.06877061, -0.0019576654, 0.031322647, 0.03299319, -0.026923556, 0.010225104, -0.0073991055, 0.02579594, 0.024278533, 0.008213494, -0.0074339085, 0.018946722, 0.0028486208, -0.035916638, -0.01644091, -0.016287778, -0.022872493, 0.0010745408, -0.020840002, 0.022287805, -0.010343434, -0.00025949944, -0.0051334337, 0.032130077, 0.008095164, -0.031322647, 0.0136427535, 0.013218157, 0.0054571014, 0.015452506, 0.024626562, -0.0075383172, -0.030682273, -0.0037413163, -0.03987025, -0.018264584, 0.015549954, -0.030793643, 0.0036717104, 0.005122993, 0.0032488545, 0.0027111492, -0.026338866, 0.063313514, -0.024682246, -0.009299346, -0.0028242588, -0.042376064, -0.043963075, -0.011109099, -0.022176435, 0.023512868, -0.031656757, -0.027856275, -0.0038874885, -0.0044860994, 0.0042633605, 0.0013312126, -0.005039466, -0.008526721, 0.036946803, -0.013141591, 0.025545359, -0.026533764, -0.050088394, -0.026979242, 0.0032505947, -0.0057633673, 0.012786601, -0.012132306, -0.044185814, 0.009897957, -0.04106747, 0.008860829, 0.03157323, 0.048529223, 0.009758745, -0.0028486208, 0.015048792, 0.009612572, -0.0064211427, -0.019085933, 0.010858518, -0.014826053, -0.0108724395, -0.003539459, -0.036473483, -0.010301671, -0.046301834, 0.018612614, 0.007475672, -0.01574485, 0.021953696, -0.033104558, 0.020840002, 0.0148817375, 0.0071415636, -0.0069675487, -0.023123074, 0.063591935, -0.0046287915, -0.02341542, 0.0011163043, -0.019266909, -0.007301657, 0.004910695, 0.008686814, 0.028259989, 0.033327296, 0.008310943, -0.01088636, 0.019851599, 0.0023143957, 0.024696168, 0.027758826, 0.009097489, 0.019364357, -0.018988485, 0.01170771, -0.020770395, 0.0028120778, -0.007047596, 0.020324918, 0.02745256, -0.026784344, -0.0062958524, -0.0001998994, -0.0065707956, 0.0013068505, 0.0042877225, 0.00934111, -0.0028103376, -0.03204655, 0.0029182266, 0.008596326, -0.008561524, -0.009612572, -0.019099854, 0.0076496867, 0.0043956116, 0.006664763, -0.033884145, 0.031322647, -0.004298163, 0.0026467638, 0.011791237, 0.018863194, 0.030598747, 0.00438169, -0.011700749, 0.018236741, -0.00874946, 0.010273828, -0.00095795095, -0.0018236742, 0.027201978, -0.0044443356, 0.013420015, -0.013294724, 0.03705817, -0.007040635, -0.0050359857, -0.029902687, -0.025879467, -0.0050916704, -0.008812105, -0.02696532, 0.013433936, 0.0051647564, -0.016078958, 0.00029538997, 0.007419987, -0.004367769, -0.027410798, -0.0077471347, 0.0017218755, 0.0272159, 0.04532735, -0.03207439, 0.004332966, 0.013176394, -0.009494242, 0.019921204, -0.0134478565, -0.00366823, -0.010587055, -0.02200938, 0.01950357, 0.016357383, -0.010865479, 0.008665932, -0.016120723, 0.003918811, 0.012710035, 0.008150849, 0.0034681133, 0.02153606, -6.063325e-05, 0.038728714, 0.010795873, -0.024849301, -0.019197302, -0.04103963, -0.03062659, 0.050004866, 0.014269206, -0.0001604198, 0.005596313, 0.0022308687, -0.010245986, -0.019851599, 0.01100469, 0.0044652172, -0.014867817, -0.016635805, -0.0046670744, 0.016830703, 0.0133016845, -0.037726387, 0.009153173, 0.007844583, 0.020394525, -0.0050673084, 0.027939802, -0.012243675, 0.013893334, -0.025768097, 0.07344813, -0.002023791, 0.0034594124, -0.0070998003, -0.0030661393, 0.0098770745, 0.012508177, -0.026672974, -0.019781992, -0.016607964, -0.07578689, -0.013907256, -0.008192613, -0.011895645, 0.01135968, -0.058970105, -0.0260326, 0.003967535, 0.010684503, 0.008665932, 0.046385363, -0.013976862, -0.04541088, -0.032714766, 0.024348138, -0.01680286, 0.035081364, -0.025030276, -0.040176515, -0.006838778, 0.026116127, 0.0010806314, 0.012744837, -0.020157864, -0.0041485108, 0.008672893, -0.017401472, -0.034413148, 0.012960616, -0.024556955, 0.034023356, -0.013468739, 0.016761096, 0.001253776, -0.037225228, 0.01798616, 0.022287805, 0.0058120913, 0.017902633, -0.008728578, 0.0009692619, -0.00028059873, -0.023192681, -0.02792588, 0.00065125007, -0.032686923, -0.005213481, -0.0025875987, -0.022844652, 0.008150849, 0.013002379, -0.0153829, 0.043489758, -0.0036369073, -0.011046453, -0.0057076826, -0.0023822614, 0.0016366084, -0.0019054611, 0.0048445696, 0.05420906, -0.014206561, 0.009737863, -0.04031573, 0.035164893, -0.013371291, 0.021744879, 0.004319045, 0.027285507, -0.015508191, 0.003703033, 0.014491945, 0.021981537, -0.011540656, -0.009403755, 0.015355058, 0.02887252, 0.037642863, -0.0051403944, 0.007872425, -0.019252988, 0.0061810026, -0.01407431, 0.026338866, -0.014631157, -0.009487282, -0.02317876, -0.008136928, -0.00075261365, 0.020171786, 0.04543872, -0.0058642956, -0.019684544, -0.021619588, 0.0045696264, 0.00013105483, -0.0063097733, -0.001005805, 0.00014203951, -0.006657803, 0.018974563, 0.028844679, -0.01266131, 0.004165912, -0.028482728, -0.02354071, -0.03182381, -0.027048847, 0.025099881, 0.004305124, 0.011554576, -0.009800509, -0.053568687, -0.0018323748, -0.009201898, -0.017819107, 0.023164839, 0.04165216, -0.012821404, -0.019726308, -3.504765e-05, -0.011540656, -0.021577824, 0.02638063, -0.05039466, 0.020185707, -0.007204209, 0.009793548, -0.007134603, 0.016886387, -0.00709632, -0.016788939, 0.007607923, -0.018584771], "content_hash": "a5efe25b4805f10d2c5aa1847ae8ba66"}, {"content": "# Experiment: Exploring Attention Patterns\n\n## Goal\n\nLearn to read attention visualizations and understand what different attention head categories reveal about how the model processes text.\n\n## Prerequisites\n\nComplete \"Your First Analysis\" first so you're familiar with the basic workflow.\n\n## Steps\n\n### Step 1: Run an Analysis\n\n1. Select **GPT-2 (124M)** and enter the prompt: `The cat sat on the mat because it was`\n2. Click **Analyze**.\n3. This prompt is ideal because it contains a pronoun (\"it\") that needs to be resolved -- the model must figure out that \"it\" refers to \"the cat.\"\n\n### Step 2: Open the Attention Stage\n\n1. Expand **Stage 3 (Attention)** in the pipeline.\n2. Look at the **head categories** section at the top.\n\n### Step 3: Explore Previous-Token Heads\n\n1. Click on **\"Previous-Token\"** to expand the category.\n2. Note which heads are listed (e.g., L0-H3, L1-H7).\n3. In the **BertViz visualization** below, double-click on one of those head squares. This will show only that head's attention pattern.\n4. **What to look for**: You should see a strong diagonal pattern -- each token attends heavily to the token directly before it. Lines should mostly connect each word to the previous word.\n\n### Step 4: Explore First/Positional Heads\n\n1. Click on **\"First/Positional\"** to see which heads focus on the first token.\n2. Double-click one of those heads in BertViz.\n3. **What to look for**: You should see many tokens sending attention lines to \"The\" (the first token). This is a common pattern -- the first token acts as a \"sink\" for attention when there's no better target.\n\n### Step 5: Explore Bag-of-Words Heads\n\n1. Find a **\"Bag-of-Words\"** head in the categories.\n2. View it in BertViz.\n3. **What to look for**: Attention should be spread broadly and evenly across many tokens. Lines will be thin and numerous rather than thick and focused. This head is gathering a general summary of the whole input.\n\n### Step 6: Look for Interesting Patterns", "source_file": "experiment_exploring_attention.md", "chunk_index": 0, "embedding": [-0.002631302, 0.050220467, 0.008488709, -0.018690977, -0.037197415, -0.03429755, -0.0045903614, 0.03498297, 0.018730521, 0.051802214, 0.030554079, -0.0130889565, 0.0045046834, -9.92711e-05, 0.0035984744, 0.02215764, 0.0069465055, 0.022302633, 0.012508982, 0.06316443, 0.012186042, -0.007111271, 0.014617979, 0.051459502, 0.02065498, -0.02559794, -0.008330534, 0.049956843, 0.013721655, 0.002954242, 0.018532801, -0.007579204, -0.053515773, -0.015382489, -0.018730521, 0.030606804, -0.052461274, 0.0320831, -0.030422267, -0.00024673605, 0.006834465, -0.00032664722, -0.025031146, 0.029710481, -0.0130362315, 0.017386036, 0.03885825, 0.013438258, -0.030290455, 0.049587768, -0.027838746, 0.034719344, -0.0043662805, 0.009997959, 0.009997959, -0.0032277524, -0.05362122, 0.021867651, 0.010479074, 0.008205312, 0.03239945, 0.018967783, 0.04318169, 0.01796601, -0.0025571575, -0.02831327, -0.058946438, -0.007935097, 0.01722786, 0.0009869442, 0.065695226, 0.023264863, -0.020364992, 0.0069069616, 0.021920377, -0.02337031, -0.0055756583, 0.003562226, -0.005114315, -0.026428355, 0.023198955, 0.0008905565, -0.0017250928, -0.013537118, -0.013490983, -0.032214914, -0.047531497, -0.04948232, -0.04866508, -0.05467572, -0.00042385876, 0.028893245, -0.03777739, 0.022684887, 0.024780702, -0.010577933, -0.013484393, -0.01366893, -0.012021277, 0.022539895, 0.048585996, 0.009101636, -0.016305175, 0.006060068, 0.04724151, -0.01775511, 0.034508448, 0.00078675436, -0.014407079, -0.046345185, -0.07476391, 0.008317353, 0.016173363, 0.023409855, 0.051274963, 0.022078551, -0.02549249, 0.01797919, 0.03432391, -0.034060284, 0.0003320021, 0.020378174, 0.019007325, 0.016015189, -0.014103911, -0.027469672, 0.020589072, -0.02110314, 0.0019294018, -0.007935097, 0.005552591, 0.028286908, 0.030738616, -0.03316396, 0.0025011373, -0.041178145, -0.0029048123, 0.025307951, -0.066538826, 0.010057274, 0.008330534, 0.0063731223, 0.0019672979, 0.046028838, -0.035984743, -0.035035696, 0.0011871341, 0.07196949, -0.03487752, 0.015619751, 0.0032722391, 0.023554849, -0.028497808, -0.053990297, 0.0029509468, -0.03337486, 0.004165267, -0.0022506942, 0.00046587392, -0.044499815, 0.061688133, -0.015962463, -0.012027867, 0.003605065, -0.042364456, 0.046266098, -0.0391746, 0.010235221, -0.0034106418, -0.04529069, -0.007190358, -0.018374627, 0.04175812, -0.035773844, 0.050009567, 0.0073221703, -0.08267264, 0.008679836, -0.030554079, -0.051169515, -0.0016369433, -0.024582984, -0.017689204, -0.008653474, -0.038515538, -0.045686126, 0.01560657, 0.012528754, -0.026850155, 0.0039214143, -0.008356896, 0.010386805, 0.06242628, 0.0070980894, -0.0025752818, -0.029710481, 0.06253173, 0.019745475, 0.019653207, 0.02056271, 0.023633936, 0.020536348, 0.03656472, 0.0033282593, 0.028181458, 0.030369543, -0.011981733, -0.005582249, -0.062900804, -0.05810284, 0.01616018, -0.059210062, 0.03358576, 0.016002007, -0.06400803, -0.019837743, -0.04692516, 0.012456258, 0.031476766, 0.0685951, 0.051828574, 0.007935097, 0.044209827, 0.04671426, -0.012759426, 0.043998927, 0.077400155, -0.029868655, 0.020299086, -0.034772072, 0.02206537, 0.008297581, 0.011830149, -0.0046002474, -0.0074276202, -0.072444014, 0.055993844, 0.0052889665, 0.01205423, -0.00024405861, 0.01387983, 0.01701696, 0.009048911, 0.018216453, 0.037091967, -0.0026675505, -0.020839516, -0.065695226, 0.03595838, 0.054833896, 0.038726438, 0.001738274, -0.072074935, 0.017741928, -0.024767522, 0.002132063, -0.023633936, 0.019547757, -0.008897327, -0.008528252, -0.013998461, 0.010024321, -0.0015281982, -0.018730521, -0.0016221144, -0.048902344, -0.023541667, -0.023950286, -0.026349269, -0.04028182, -0.041810844, 0.012574889, 0.037276503, 0.020114549, -0.06421892, -0.009299354, -0.018414171, -0.014578435, 0.0027351042, -0.0057667857, -0.03777739, -0.008851192, 0.05541387, 0.041942656, -0.026415175, 0.014117092, -0.0015133694, -0.029183231, -0.000318203, -0.00040099758, 0.044157103, -0.042232644, 0.057206515, -0.012792379, -0.019798199, -0.00775056, 0.0130823655, 0.019218225, -0.010584524, 0.017201498, 0.029578669, -0.03904279, 0.038357362, 0.0043432135, 0.02431936, 0.024662072, 0.021709478, -0.023225319, 0.026573349, 0.013471211, -0.015487939, -0.042891704, 0.009655247, -0.028972331, -0.026863337, -0.006953096, 0.007730788, 0.048717808, -0.0048836437, 0.004297079, -0.03079134, -0.034692984, 0.053542133, 0.03195129, 0.023172593, 0.0035589307, -0.013761199, 0.0148025155, 0.026177913, 0.0036215414, 0.036643803, -0.02077361, -0.009246629, 0.013359171, -0.02775966, -0.021643572, -0.026665617, 0.043735303, 0.011718109, 0.056152016, -0.0806691, -0.009292764, -0.0018008848, 0.013194406, -0.05348941, 0.06906962, -0.0014903023, -0.012845104, -0.005908484, -0.030949516, -0.036960155, 0.0038357363, 0.016529256, -0.03672289, -0.0056217923, 0.057206515, -0.031819478, -0.025123414, 0.011961961, -0.033954836, -0.0043728715, -0.0012546878, 0.04971958, 0.005424074, 0.030448629, 0.0071178614, -0.030923152, 0.0064917533, 0.0066367467, 0.0022688182, 0.018466895, 0.02733786, -0.007697835, -0.03261035, -0.043418955, 0.001234916, 0.005242832, -0.012904419, -0.035009332, 0.07175859, -0.010037502, 0.034139372, -0.020272724, -0.044288915, 0.029394131, 0.025307951, 0.0031223027, 0.0075528417, -0.060844533, -0.008422802, -0.019363219, -0.021722658, -0.044710714, 0.038805526, -0.015369308, 0.031450402, -0.012008096, 0.030633166, 0.052935798, 0.05799739, 0.018730521, -0.019534575, -0.015145227, -0.0037533538, 0.0052131745, 0.0053977114, -0.0037138101, 0.02786511, 0.012614432, 0.0059249606, 0.05457027, -0.010881101, 0.018770063, -0.020707704, -0.023726204, 0.027522398, 0.019244589, 0.022012645, 0.012311264, 0.013510755, -0.016911512, -0.05035228, -0.017386036, 0.0038686895, -0.07144224, 0.010874511, 0.038594626, -0.030527717, 0.014341173, -0.038568262, -0.004488207, 0.044631626, 0.04120451, -0.0022506942, -0.022328995, -0.021801746, 0.033743937, 0.073287606, 0.007783513, -0.054622997, 0.00048811722, -0.014907965, -0.044183467, 0.00017918227, -0.02131404, 0.032636713, -0.035483856, -0.05889371, 0.030527717, -0.03798829, -0.005760195, 0.0045145694, 0.0068805995, 0.02464889, -0.008963233, 0.0027581712, 0.039095514, -0.021221772, 0.076240204, -0.025742931, 0.01668743, 0.011922418, -0.013893011, 0.027944196, -0.0028389064, 0.017346492, -0.020219998, -0.014907965, 0.05072135, -0.026230637, 0.023515305, 0.006920143, 0.012541936, -0.046028838, -0.0075001167, -0.032056738, -0.000750506, 0.00135849, -0.021169048, -0.045607038, 0.016911512, 0.0031750274, 0.03442936, 0.047320597, 0.007730788, 0.03722378, -0.04283898, -0.005941437, 0.028814157, 0.003221162, -0.03445572, -0.012192633, 0.011480846, -0.014433441, 0.0048408047, -0.019060051, -0.0113292625, -0.030185005, 0.007823057, -0.0013980337, 0.01647653, -0.030211367, 0.008376668, 0.03366485, -0.005542705, 6.60091e-05, 0.030000467, 0.013840286, -0.019033689, 0.045396138, -0.035035696, 0.010182496, 0.0076385196, 0.034192096, -0.014433441, 0.020826336, 0.02723241, 0.025057508, 0.0032887156, 0.0027334564, -0.0017547505, 0.015303402, -0.0056876983, -0.008172359, 0.00025209092, -0.038146466, -0.03195129, -0.06300625, -0.04679335, -0.008066909, 0.020615436, -0.016661068, -0.0045046834, -0.007243083, -0.032214914, -0.007783513, -0.0036643804, -0.038515538, -0.06158268, 0.010485664, -0.0051176106, -0.0034172325, 0.02184129, 0.0030613395, 0.010933826, 0.0028389064, 0.023844836, -0.002141949, 0.020312266, 0.039780937, 0.006587317, -0.04283898, 0.02400301, -0.0062248334, -0.020786792, -0.0064686863, -0.028207822, -0.04057181, -0.0046958113, -0.041995384, 0.013378943, -0.028049646, 0.0043761665, -0.03756649, -0.013721655, 0.04845418, 0.01819009, -0.043128967, 0.0026873222, -0.009127998, 0.018203272, -0.012957144, 0.014038004, -0.035668395, 0.028708708, -0.0043333275, -0.05499207, 0.013761199, -0.01237717, 0.020272724, -0.038331002, -0.0017679317, -0.022091733, 0.018203272, 0.00071260997, -0.02087906, 0.04262808, -0.005015456, -0.02227627, -0.027627846, 0.051169515, -0.019218225, -0.006274263, 0.036248367, -0.046450637, -0.022513531, 0.0113029005, -0.014552072, -0.029974105, 0.028866882, 0.02669198, 0.01259466, -0.009411395, 0.01366893, -0.026191093, 0.018638251, -0.041705396, -0.006953096, 0.015817469, 0.023713022, -0.035114784, -0.017781472, -0.027706934, -0.018862333, -0.028603258, 0.03466662, -0.030554079, 0.0035062057, -0.019468669, 0.0024204024, 0.01539567, -0.017465122, 0.01086792, -0.016331537, 0.023185775, 0.020852698, 0.011849921, -0.0049462547, 0.016595162, -0.0010561456, -0.029473219, 0.015791107, 0.03593202, -0.018427351, -0.02194674, -0.029104143, 0.010900873, -0.005252718, 0.011065639, 0.0078428285, 0.0007167291, -0.010960189, -0.025637481, -0.006719129, 0.025479307, 0.0069596865, 0.0130889565, 0.00095151964, -0.029578669, -0.02733786, -0.043577127, -0.023765748, 0.08145997, 0.005443846, 0.029104143, -0.022513531, -0.025044328, -0.035773844, -0.007473754, 0.018651433, -0.032768525, 0.027917834, -0.0060864305, -0.025057508, -0.032873973, -0.011856511, 0.014288448, 0.015896557, 0.01356348, 0.005951323, -0.09015958, 0.0064192563, 0.032847613, 0.004689221, 0.020075005, 0.012027867, 0.009312536, -0.038884614, 0.0021452443, 0.033401225, 1.8034078e-05, 0.020496804, -0.02926232, 0.0010890987, 0.0038126693, 0.0022671707, 0.011513799, -0.02377893, 0.0130955465, -0.013181224, 0.018901877, -0.04455254, 0.014710247, -0.0011442951, -0.0012077297, 0.0006487634, 0.0004992389, 0.038673714, -0.024042554, -0.10903509, -0.015474758, -0.011401759, -0.01065702, 0.012621023, 0.001647653, -0.011856511, 0.011685155, 0.056573816, 0.0130494125, -0.02582202, -0.049904115, 0.005717356, 0.0063335784, 0.036327455, -0.021959921, 0.016713792, -0.019112775, -0.006686176, 0.0058953026, -0.01969275, -0.033928473, 0.027416948, -0.0008477175, 0.007526479, 0.023963466, -0.015870195, -0.006422552, -0.023515305, 0.0056712218, 0.048480544, 0.028365996, 0.022645345, 0.0016970827, -0.07054591, 0.0048506907, 0.019073233, -0.033322137, 0.022025827, 0.00425424, -0.02066816, 0.020997692, -0.0018717339, 0.00980024, 0.007566023, 0.01743876, 0.010129771, -0.01270011, 0.037276503, -0.032346725, -0.017900104, -0.0074078483, 0.01152039, -0.015382489, -0.015461576, 0.020101368, -0.008343715, 0.002061214, -0.019086413, 0.0009844727, 0.032214914, 0.017267404, 0.00022778804, -0.017135592, 0.008917099, -0.030633166, -0.007559432, 0.0318722, 0.020167274, 0.022434445, 0.02248717, 0.022869425, 0.0072694453, 0.03764558, -0.010808604, 0.036459267, -0.015052958, 0.04993048, 0.011230404, 0.012251948, 0.006218243, 0.014156635, 0.0062446054, -0.031608578, -0.027680572, 0.036090195, -0.0032607054, -0.034060284, -0.009543207, -0.019679569, -0.010314308, -0.008014184, -0.036116555, 0.01452571, -0.015408852, 0.01366893, 0.0024319359, -0.022342175, -0.08884145, -0.004089475, 0.008706199, -0.02454344, -0.02873507, -0.018875513, 0.028234184, 0.0059710946, -0.018770063, -0.00527249, 0.014446623, -0.02831327, -0.0113226725, -0.019679569, 0.008838011, -0.035457496, -0.008910508, -0.004082884, -0.006705948, -0.0010396692, -0.031450402, 0.014907965, 0.0014276914, 0.0015430271, -0.035141144, -0.0028520876, -0.018111004, 0.002530795, 0.029763205, 0.03216219, -0.030343179, 0.035879295, -0.015316583, 0.02346258, 0.0020299086, 0.033401225, -0.021933557, 0.04908688, 0.040070925, -0.033216685, 0.0063335784, -0.010861329, 0.0028899836, -0.025110234, 0.04811147, 0.0050121606, -0.013471211, -0.014683885, -0.021010872, 0.032452174, -0.004649677, 0.019943193, 0.01764966, 0.023212137, 0.018796427, 0.025980193, -0.022210363, -0.018308721, 0.02723241, 0.03743468, -0.04552795, 0.0014079196, 0.05509752, 0.005931551, -0.0043761665, 0.030343179, -0.015158408, 0.022302633, -0.041072696, -0.026217457, -0.030158643, -0.009385032, 0.0040202737, -0.012337626, -0.017161954, -0.015659295, 0.0033546216, 0.028708708, -0.012113545, -0.0025291475, 0.043998927, 0.0012686929, -0.009002777, 0.017241042, 0.017992372, 0.020417716, 0.03002683, -0.016436987, -0.014683885, 0.053436685, 0.005552591, -0.0031799704, 0.021656752, 0.019890469, 0.0004839981, -0.021775384, 0.01452571, 0.033005785, 0.058155563, 0.009483892, 0.006854237, 0.034271184, -0.018097822, -0.013009869, -0.055993844, -0.00047287645, 0.020219998, -0.014907965, -0.0025653958, -0.045712486, 0.010070455, 0.022922149, 0.029209593, 0.008225084, 0.009193905, 0.020167274, 0.061529957, 0.00039461293, 0.0021287678, -0.020865878, -0.0015537369, 0.01539567, -0.012548526, -0.0011533572, -0.027496034, -0.012726473, -0.038515538, -0.00871279, 0.0035754072, -0.012363989, 0.020061824, 0.004339918, -0.001277755, -0.017834198, 0.007262855, -0.0039774347, -0.0024863086, 0.027838746, -0.01431481, 0.01616018, -0.006554364, 0.016094275, 0.031371314, 0.022856243, -0.004175153, 0.0046760393, -0.00775056, -0.028603258, 0.0077241976, 0.027074236, 0.019798199, 0.00074597495, 0.027917834, -0.007196949, -0.016661068, -0.009984777, 0.008225084, -0.010512027, -0.005770081, 0.023647116, -0.004613429, 0.018045098, 0.06037001, 0.003115712, 0.030949516, -0.017900104, -0.015712019, -0.013233949, -0.012416714, 0.019020507, -0.02559794, 0.04679335, 0.032346725, 0.032768525, 0.011480846, 0.03543113, 0.0069069616, -0.010314308, 0.02044408, 0.033849385, 0.001106399, 0.021024054, -0.026771067, 0.0055229333, 0.019824563, -0.010142952, -0.012983507, 0.013589843, -0.014683885, -0.0006289916, -0.010281355, 0.03904279, -0.03429755, 0.0016509484, -0.008600749, -0.022698069, 0.025360676, 0.010624067, -0.0030580442, -0.0013288322, -0.028998695, 0.028392358, 0.021182228, -0.035562944, 0.02248717, 0.021551302, 9.067241e-05, -0.0017036733, -0.010241811, -0.025466125, -0.004910006, 0.01797919, -0.042496268, -0.0015471462, -0.019139139, 0.0036413134, -0.005951323, 0.026006557, 0.013629386, -0.014354354, -0.06553705, -0.026942423, -0.031371314, 0.003832441, -0.011856511, 0.023185775, 0.035589308, -0.026256999, 0.017702384, -0.006307216, -0.017043324, -0.011869693, -0.011355625, 0.014723428, -0.017662842, 0.010624067, 0.011988324, 0.020088186, 0.029104143, 0.012647385, 0.037197415, 0.011698337, 0.008950052, -0.015250677, 0.012746245, -0.02496524, 0.033743937, -0.0069069616, 0.0067224246, 0.003993911, -0.009721153, 0.02775966, 0.024609346, -0.0020908718, 0.011790605, -0.008772105, 0.025887925, 0.024464354, 0.008969824, 0.012304673, 0.018572345, 0.036828343, -0.014749791, -0.048612356, -0.010643839, 0.0024335836, -0.02765421, -0.012548526, 0.009398214, 0.013497574, 0.0045475224, 0.022131277, -0.019903649, 0.016924692, 0.0039906157, -0.011210632, 0.013906192, -0.02066816, 0.015777927, -0.008244856, 0.0038719848, 0.012113545, 0.012021277, -0.013748017, 0.012581479, 9.314389e-05, -0.056362916, -0.032689437, 0.0031931517, -0.020325448, 0.0065411828, -0.02346258, 0.007019002, -0.01496069, -0.016489713, 0.0017399216, 0.0034567763, -0.0021386538, 0.008014184, -0.0053318054, 0.015540664, 0.005605316, 0.042285368, -0.039359137, -0.010749289, 0.03358576, 0.012449667, -0.06116088, -0.013286674, -0.002759819, 0.0130164595, 0.0012217348, -0.0073683048, 0.03195129, -0.011803787, -0.017306948, 0.016436987, -0.0078757815, 0.010769061, -0.03485116, -0.01797919, 0.03701288, -0.0159361, 0.008442574, 0.009068683, -0.0046002474, -0.0016567152, -0.0078494195, 0.011830149, -0.012528754, 0.033533037, 0.0021452443, 0.018111004, 0.01356348, -0.027786022, 0.0056745172, -0.025690207, -0.02077361, 0.029341407, 0.015672477, -0.0008374197, 0.007598976, -0.013682111, 0.010182496, 0.034903884, 0.0025242046, 0.011474256, -0.014987052, -0.021709478, 0.043972567, 0.06880599, -0.03013228, -0.038805526, 0.014143454, 0.01981138, 0.0131482715, -0.013919373, -0.007954869, -0.032030378, -0.006748787, -0.023001237, -0.0059875716, -0.02077361, 0.0014655874, -0.0060633635, -0.015751563, -0.019666387, 0.0095497975, 0.0639553, 0.014235723, 0.004132314, 0.05551932, -0.034350272, -0.015553845, 0.027996922, 0.035668395, 0.02443799, 0.010993142, 0.0096420655, 0.016555618, 0.023950286, -0.019956375, -0.027469672, -0.01258807, -0.021129504, 0.016081095, -0.006521411, 0.011006323, -0.021986283, 0.008304171, 0.004972617, 0.035246596, 0.018559165, -0.006188585, 0.019244589, 0.026072463, 0.004382757, 0.027996922, -0.017886922, -0.021933557, -0.027206048, 0.026467899, 0.0043003745, -0.03690743, 0.014591616, -0.029499581, -0.0065708407, 0.01796601, 0.018427351, 0.007664882, -0.010993142, 0.029868655, -0.032478537, 0.009945234, -0.0020579188, -0.04703061, -0.016964236, -0.0006759497, -0.013550299, 0.0130494125, -0.014578435, -0.021155866, -0.018361446, -0.0005614378, 0.032900337, 0.015013415, 0.008027365, -0.00096305326, 0.03788284, 0.0006261082, -0.010558161, -0.0025999965, -0.035009332, 0.0038818708, -0.0013939146, -0.004221287, 0.031503126, -0.028076008, -0.015145227, 0.028102372, -0.02831327, 0.031265866, 0.030264093, 0.050826803, 0.022500351, -0.0056844032, -0.0029838998, -0.027680572, 0.0054405504, -0.025901107, -0.0027136847, -0.01107882, -0.01227172, 0.0047584223, -0.008620521, -0.012851694, -0.03606383, 0.025861563, 0.0049495497, 0.0015240791, 0.006445619, -0.009213676, -0.00061333884, -0.026467899, -0.0041422, 0.01710923, -0.010050684, 0.042074468, -0.038805526, -0.035457496, 0.030422267, -0.012449667, 0.0070849084, -0.0019837744, -0.000104317034, -0.0022391605, 0.020734066, -0.006317102, -0.032768525, -0.006261082, -0.012785788, 0.014591616, 0.0043333275, -0.03295306, 0.00462661, 0.018532801, -0.0096025225, -0.02422709, -0.06348078, 0.009029139, 0.034719344, 0.0060996115, -0.00914777, -0.017741928, 0.014670704, -0.0055031613, 0.00808009, 0.0058096247, -0.0041158376, 0.028207822, -0.015817469, -0.023027599, 0.015303402, 0.018980963, -0.003756649, -0.033954836, -0.018335083, -0.0025999965, -0.024727978, -0.0050418186, 0.026296543, 0.0013189462, -0.010683383, 0.0096091125, 0.020285904, 0.03400756, -0.02905142, 0.016858786, 0.038700078, 0.03680198, 0.0042476496, -0.017359672, -0.006254491, 0.010518617, -0.004478321, 0.0038291458, 0.029631393, -0.002992138, 0.0061292695, 0.025242046, 0.0016155238, 2.0106516e-05, -0.03522023, -0.041151784, -0.0008946756, 0.027100598, -0.0048935297, -0.021775384, -0.014064367, -0.0033513263, 0.0041619716, -0.03522023, -0.003581998, -0.0113819875, 0.027390584, 0.056415644, 0.0072035394, -0.0013453087, 0.026072463, 0.011250176, -0.020681342, -0.02691606, -0.02658653, -0.01712241, -0.02066816, 0.031160416, -0.02108996, -0.002242456, 0.0017217975, 0.008594158, -0.020206818, -0.024569804, -0.037039243, -0.0038027833, 0.0066169747, 0.011671974, 0.009523435, 0.0148552405, 0.015211133, -0.018045098, 0.021485396, -0.01764966, 0.026665617, -0.0065741357, -0.032030378, 0.031424038, -0.022843063, -0.031898566, 0.006294035, -0.010722926, 0.0096157035, -0.037302867, 0.024477534, 0.023409855, 0.008152587, 0.010261583, -0.019547757, 0.017478304, 0.00043580425, -0.0010767413, -0.0048605767, 0.04423619, 0.023923922, 0.02033863, -0.008145997, 0.034271184, -0.008244856, -0.0074671637, -0.008126225, -0.036222007, 0.026731525, -0.002982252, -0.029578669, -0.037830114, -0.013721655, -0.03777739, -0.0113226725, 0.007249674, 0.0025110233, 0.007902144, -0.04658245, -0.005295557, -0.023343949, 0.0015619751, 0.025242046, 0.025017964, 0.008093272, -0.015646113, -0.045923386, 0.006379713, -0.02733786, -0.013695292, -0.027496034, 0.0040960656, 0.0022012645, 0.041942656, 0.01442026, 0.0077439696, -0.016713792, 0.0017975895, 0.016726974, -0.03595838, 0.005219765, -0.0113688065, 0.004007092, -0.0003268532, 0.008343715, -0.011289719, 0.023554849, -0.0032689436, 0.014341173, 0.037487403, -0.010512027, -0.01658198, 0.03216219, -0.005466913, -0.014235723, -0.0025588053, -0.043735303, 0.028998695, -0.026098825, -0.023660298, 0.019943193, -0.012126727, 0.01722786, 0.0049792076, 0.033823024, 0.04120451, -0.027074236, -0.0054306644, -0.0068212836, -0.011685155, -0.009780468, -0.009220267, 0.0076055666, -0.00425424, -0.022039007, -0.026401993, 0.008673246, 0.021973101, 0.0046529723, 0.026573349, 0.018440533, -0.0069135525, 0.032979425, -0.041494496, -0.0029806045, 0.0069267335, 0.037039243, -0.0031882087, 0.013603024, 0.031687666, 0.056099292, 0.023198955, 0.011955371, -0.004995684, -0.024148004, 0.0042904885, 0.014776153, -0.010584524, -0.013893011, -0.02130086, -0.01840099, -0.0049133017, 0.0012003153, 0.03962276, 0.002797715, -0.01637108, -0.016397443, 0.007823057, 0.015224314, -0.0012225586, -0.00446514, -0.034613896, -0.008963233, 0.019363219, 0.015659295, 0.0058063297, -0.011098592, -0.021485396, -0.05509752, 0.017201498, -0.019191863, -0.0060732495, -0.02227627, -0.0019162205, -0.023080325, -0.019218225, 0.009589341, 0.0033183733, 0.009312536, 0.0030695777, 0.027179686, 0.020114549, -0.03551022, -0.031634938, -0.036116555, -0.046635173, 0.033427585, -0.017741928, 0.013174634, -0.028076008, 0.018875513, -0.007071727, 0.009734334, -0.009154361, -0.02862962, 0.037487403, -0.0024896038], "content_hash": "e9bbc428355b61182f95ca4efa7383c8"}, {"content": "categories. 2. View it in BertViz. 3. **What to look for**: Attention should be spread broadly and evenly across many tokens. Lines will be thin and numerous rather than thick and focused. This head is gathering a general summary of the whole input. ### Step 6: Look for Interesting Patterns\n\n1. Now single-click to select multiple heads from different categories.\n2. Look for heads where the token \"it\" attends strongly to \"cat\" -- this would suggest the head is helping resolve the pronoun reference.\n3. Try hovering over the word \"it\" on the left side of BertViz to see which words it attends to most strongly.\n\n### Step 7: Try Different Prompts\n\nRun the analysis again with different prompts and compare:\n- `Alice gave the book to Bob because she` (pronoun resolution: does \"she\" attend to \"Alice\"?)\n- `The dogs in the park were` (subject-verb agreement: does the model connect \"dogs\" to \"were\"?)\n- `1 2 3 4 5` (number sequence: what patterns emerge with non-natural-language input?)\n\n## What You Should Learn\n\n- Different attention heads serve different purposes\n- Previous-Token heads are the easiest to identify visually (strong diagonal pattern)\n- The same prompt can reveal different patterns in different heads\n- BertViz is a powerful tool for understanding attention, but it takes practice to read fluently\n- Not all heads have obvious patterns -- and that's okay. The \"Other\" category captures complex, context-dependent behavior.", "source_file": "experiment_exploring_attention.md", "chunk_index": 1, "embedding": [0.004536667, 0.04139891, 0.027292939, -0.01295357, -0.02277086, -0.024433818, -0.013617295, 0.033725962, 0.008460665, 0.038656488, 0.04778817, 0.009569304, 0.024258768, 0.011254143, -0.030195821, 0.004303269, 0.0015763459, 0.010116329, 0.0031417513, 0.06850221, 0.02277086, 0.027730558, 0.022129016, 0.041077986, 0.025527868, 0.022785446, -0.017825745, 0.02866415, 0.0298749, 0.009605772, -0.0023759152, -0.016760869, -0.024667215, -0.03594324, -0.03025417, 0.026082188, -0.024404643, -0.01077276, 0.007421316, 0.014456067, 0.010298671, 0.014142439, -0.028066067, -0.011793875, -0.01747565, 0.025104836, 0.0349513, -0.016191963, -0.035213873, 0.052456122, -0.042011578, 0.039590076, 0.017606936, 0.02838699, -0.002727835, -0.019284481, -0.0524853, 0.005725536, 0.031975478, -0.0137923425, 0.018453002, -0.023004256, 0.022639573, 0.037664548, -0.0070894537, -0.033755135, -0.06867726, -0.020626517, 0.021210011, -0.031333636, 0.06377591, 0.05298127, 0.017067203, 0.020247247, 0.021705981, 0.031596206, 0.01281499, 0.015097911, 0.0135516515, -0.027905608, 0.038335565, -0.01961999, -0.0014414128, -0.03075014, -0.025163185, -0.04513327, -0.039706778, -0.030896014, -0.03836474, -0.083848104, -0.008176211, 0.026111363, -0.008416903, -0.0060245767, 0.015900215, -0.032617323, -0.002414207, -0.03235475, -0.00793552, 0.024287943, 0.04656283, -0.010415371, -0.027701383, 0.02138506, 0.07276172, -0.004201158, 0.023164717, 0.008978516, -0.002797125, -0.014251844, -0.08308957, 0.016629582, -0.014295606, 0.040757064, 0.03830639, 0.028707912, -0.025425756, -0.0014250021, 0.02894131, -0.039035756, 0.014828045, -0.0096349465, -0.0008825349, 0.024244182, -0.01697968, -0.018788511, 0.028430752, -0.02668027, -0.009532835, -0.020232659, 0.011465659, 0.014353956, 0.03731445, 0.0024087366, -0.02600925, -0.042040754, -0.008942048, 0.016775457, -0.04247837, 0.025994664, 0.00424492, -0.008343966, 0.0044819643, 0.02943728, -0.022143602, -0.0059078783, -0.01091134, 0.05403156, -0.009357788, 0.030166646, -0.0016173729, 0.0056051905, -0.046854578, -0.015812691, -0.03180043, -0.033259165, -0.0018453002, -0.026446871, -0.04247837, -0.039035756, 0.020874502, -0.031712905, -0.024069132, 0.006954521, -0.04711715, 0.021049552, -0.019153195, -0.00081233325, -0.017709047, -0.034484502, -0.028241117, -0.028532863, 0.015068736, -0.04344114, 0.008219973, -0.0035319629, -0.027074128, -0.0118595185, 0.0063090306, -0.026942842, -0.0092994375, 0.016862981, -0.0070165168, -0.0071879183, -0.0018124786, -0.0174173, -0.022858383, 0.015900215, -0.048809286, -0.0048357076, -0.030108297, 0.0065096063, 0.09884391, 0.032996595, 0.023018844, -0.045104098, 0.049363606, 0.011144738, 0.010947809, -0.009678709, 0.0026913667, 0.0070712194, 0.03605994, -0.019488703, -0.020466058, 0.013544357, -0.00396776, 0.020086786, -0.038773187, -0.04723385, 0.0051311017, -0.048488364, 0.030049948, 0.04393711, -0.079822, -0.016658757, -0.041165512, 0.008358553, -0.020291008, 0.059078783, 0.05020967, 0.008081394, 0.072936766, 0.026446871, -0.033142466, 0.056919854, 0.06902736, -0.03570984, 0.002392326, -0.047934044, 0.054381654, 0.041253034, 0.020466058, -0.0019802332, -0.039590076, -0.046942104, 0.027438812, -0.0044746706, 0.0016720754, -0.03862731, 0.00956201, -0.013267198, 0.03191713, 0.01069253, 0.008314791, 0.0034207343, -0.025775854, -0.062433872, 0.06290067, 0.038452264, 0.08425655, -0.029626915, -0.051114086, 0.010758173, -0.007045692, 0.0035501972, -0.039064933, 0.037226927, -0.017242251, 0.0037380094, -0.027220001, 0.03918163, -0.016454535, 0.02336894, -0.038014643, -0.06511795, -0.021866443, -6.854916e-05, -0.031041889, -0.025031898, -0.09079169, 0.0015097911, 0.046358608, 0.016862981, -0.05131831, 0.006217859, -0.012545124, -0.009292144, -0.0058604693, -0.019094845, -0.031712905, 0.0008811673, 0.043003518, 0.017957032, -0.024390055, 0.010284084, 0.0008784322, -0.0034097938, 0.0054483763, 0.015973153, 0.02259581, -0.040465318, 0.03810217, -0.018088318, -0.028182767, -0.0028317699, 0.051347483, 0.01957623, 0.028737087, 0.010860285, 0.028139004, -0.029189294, 0.04145726, 0.015564706, 0.02888296, 0.008752412, 0.023500226, -0.021210011, 0.01956164, 0.005251447, -0.018438416, -0.044258032, -0.022085253, -0.028459927, -0.033142466, -0.059574753, 0.023179304, 0.017315188, 0.015987739, 0.013675644, -0.03273402, 0.0052295662, 0.021064138, 0.05368146, -0.0075416616, -0.0052550943, -0.021355886, 0.024608865, 0.010181973, -0.006053752, 0.011122857, 0.0020677573, -0.006567956, 0.060683392, -0.036906004, -0.051347483, -0.012931689, 0.036526732, -0.0073410855, 0.048430014, -0.06050834, 0.0067502977, 0.008249149, 0.032383926, -0.05776592, 0.07031104, -0.015229197, 0.013909042, -0.0110863885, 0.02166222, -0.020568168, -0.004197511, 0.023427289, -0.028591212, -0.002853651, 0.08098899, -0.05875786, -0.0422158, -0.007490606, 0.0161482, 0.017038029, -0.007348379, 0.0545567, 0.04134056, 0.026490634, -0.0035264927, -0.020349357, -0.021837268, -0.021253774, -0.0065169, -0.016585821, 0.009992337, -0.0034863774, -0.006918052, -0.05881621, -0.004573135, -0.016294073, -0.0075927177, -0.016673345, 0.07071949, -0.008015751, -0.039706778, -0.027263764, -0.04828414, 0.019940913, 0.006768532, 0.013238023, -0.019765863, -0.081222385, -0.022187365, 0.0034954946, -0.012603473, -0.0011788405, 0.01912402, -0.015010387, 0.034076057, -0.039765127, 0.0422158, 0.017621523, 0.078830056, 0.036293335, 0.013194261, -0.024623452, 0.028139004, 0.020538993, -0.00878888, -0.032704845, 0.02805148, -0.00027898312, -0.0074760187, 0.047583945, -0.006403848, 0.02711789, 0.01714014, -0.009343199, 0.020261833, -0.0045549013, -0.00027989486, 0.026928253, 0.032500625, -0.015404245, -0.07381201, -0.022887558, -0.010233029, -0.048605062, -0.011742819, 0.057620045, -0.013303666, 0.026694857, -0.040202744, -0.0120053915, 0.04034862, 0.021705981, 0.0035939591, -0.013777755, -0.062025428, 0.057911795, 0.04772982, 0.03594324, -0.047525596, -0.0058385883, -0.036293335, 0.015404245, 0.0034954946, -0.024492167, 0.020626517, -0.012698291, -0.06710183, 0.038539786, -0.053593937, -0.032588147, 0.0013055681, -0.02562998, 0.047583945, 0.012559711, 0.011509422, 0.05341889, -0.005947993, 0.041982405, -0.024010783, -0.0039568194, 0.025017312, -0.0070274575, 0.012559711, 0.00084561063, 0.033025768, -0.027993131, -0.02331059, 0.04239085, -0.021034963, -0.016119026, 0.0033787957, 0.011093683, -0.061792027, -0.008190799, -0.013245317, 0.0013228906, -0.026446871, -0.0029357048, -0.05633636, 0.040086046, -0.009314025, 0.045862637, 0.027759733, -0.003032346, 0.036293335, 0.01648371, 0.020903677, 0.026257236, 0.006341852, -0.012931689, -0.024302531, 0.014514416, -0.017533999, 0.024039958, 0.0120053915, 0.0119105745, -0.043791234, 0.017665286, -0.009963162, 0.020232659, 0.002705954, -0.0041938643, 0.05292292, 0.008577364, 0.0053973207, 0.0545567, 0.0015170848, -0.020203484, 0.044958223, -0.045220796, 0.010203853, -0.0077896467, 0.04358701, -0.03092519, 0.018117493, -0.0011478424, 0.01725684, -0.011465659, 0.02281462, -0.00013949156, 0.017461061, 0.009226501, -0.009022278, -0.006137629, -0.024871437, -0.010663356, -0.008591951, -0.04414133, -0.015039561, 0.0035811954, -0.036672607, -0.0035337864, 0.0037124814, -0.028357815, 0.010670649, -0.0076364796, 0.0042303326, -0.034134407, 0.0048211203, -0.013595413, -0.007971989, 0.010517482, -0.0023248594, -0.0067502977, 0.012428425, 0.021968555, 0.007534368, 0.022741685, 0.039444204, -0.006881584, -0.019663753, 0.010532069, -0.02354399, -0.0038437676, -0.00701287, -0.021851856, -0.03180043, -0.0045184325, -0.040377796, 0.018642638, -0.040757064, 0.0034790838, -0.019386593, -0.006385614, 0.049101032, 0.025819616, 0.010561244, 0.0010028805, -0.012231496, -0.012844165, -0.023908673, 0.024754738, -0.025075661, 0.020918265, -0.020072198, -0.03428028, 0.0043215035, -0.017038029, 0.004398087, -0.029612327, -0.021297537, -0.02640311, 0.012705585, 0.022406176, -0.027205413, 0.05257282, 0.0047408896, -0.023193892, -0.018715575, 0.040115222, -0.025163185, -0.011874106, 0.030137472, -0.03273402, -0.009656828, -0.0026786027, 0.013427659, -0.03588489, 0.037781246, 0.011006158, 0.01110827, -0.020684866, 0.019780451, -0.039064933, 0.022581223, -0.012384663, -0.02287297, 0.011633415, 0.026855318, -0.034046885, -0.001685751, -0.028066067, -0.0026093128, -0.03708105, 0.005685421, -0.03025417, -0.0037380094, -0.021414235, -0.0062616216, 0.003506435, -0.016162788, 0.015827278, 0.011655295, 0.021589283, 0.025148597, 0.020363946, -0.030691791, 0.018686399, -0.029043421, -0.0349513, 0.009598479, 0.029058008, -0.01323073, -0.018555114, 0.0023649747, -0.022785446, -0.0012289846, 0.0068888776, 0.025250709, -0.013602707, 0.0198388, -0.01719849, -0.021676807, -0.002231865, -0.016717106, 0.011998098, 0.0099194, -0.03390101, -0.0075270743, -0.042565897, -0.051551707, 0.06762697, -0.0068888776, 0.021603871, -0.024842262, -0.031041889, 0.0028390638, -0.019984674, 0.033054944, -0.0057583577, 0.0054192017, 0.016556647, -0.04624191, -0.017213076, 0.0017395419, -0.00016194241, 0.03503882, -0.007556249, 0.003785418, -0.090033144, 0.014893688, 0.06196708, 0.008271029, 0.022902144, 0.017986206, 0.023514813, -0.008263736, -0.027380463, 0.025396582, -0.032996595, 0.014864513, -0.055869564, -0.030633442, 0.021268362, -0.00884723, 0.022843795, -0.022012316, -0.0008702268, 0.01648371, 0.012858752, -0.037110228, 0.034163583, -0.0015398775, -0.0087669995, -0.0018343597, 0.0035483737, 0.030137472, -0.028810022, -0.07241162, -0.022610398, -0.015535532, 0.01267641, -0.04309104, 0.020743217, -0.04145726, 0.0048247674, 0.04029027, -0.009809995, 0.002166222, -0.041690655, -0.0035903123, 0.0029594093, 0.014135146, -0.0149520375, 0.017767396, -0.009824582, -0.015345896, 0.034367803, -0.025819616, -0.02579044, 0.019372005, 0.00743955, 0.00097188243, 0.032646496, 0.0017112789, -0.003196454, -0.02965609, -0.007738591, 0.016687931, 0.023689862, 0.032879893, -0.0007161479, -0.043003518, -0.020582756, 0.015798103, -0.05557782, 0.034076057, -0.021020377, -0.032092176, 0.02832864, -0.031100238, -0.00084925746, 0.004376206, 0.02705954, -0.010984277, -0.023952434, 0.044345554, -0.038160518, -0.015593881, 0.0010821993, 0.015272959, -0.016746283, -0.0161482, 0.03180043, 0.0029120005, 0.0041136337, -0.0053973207, 0.014704052, 0.042740945, 0.017621523, -0.007468725, -0.006476785, 0.004806533, 0.0066919485, 0.0031435746, 0.003920351, -0.0007207064, 0.0029393516, 0.02138506, 0.018175842, -0.003909411, 0.03912328, -0.022479111, 0.050618116, -0.035914063, 0.025221534, 0.009489073, -0.008526308, 0.021705981, -0.005612484, -0.0099194, -0.027220001, -0.028299466, 0.01388716, 0.022289475, -0.030195821, -0.030108297, -0.0056781275, -0.014332075, -0.0070639257, -0.031712905, -0.029058008, -0.022756271, 0.0019364712, 0.009627653, -0.030137472, -0.10736292, -0.01281499, 0.017723635, -0.047000453, -0.034221932, -0.0038291803, 0.043966282, 0.017315188, -0.022129016, 0.021676807, -0.0014514417, 0.011312492, -0.010429958, -0.029568566, 0.016717106, -0.023675274, -0.021297537, -0.014171613, -0.007651067, -0.012647236, -0.009102508, -0.0012791286, 0.0023120956, 0.0035082584, -0.042186625, 0.006254328, -0.043849584, -0.0080449255, 0.01902191, 0.005878703, -0.052222725, 0.009766233, -0.01259618, 0.030108297, 0.016016914, 0.017242251, -0.011152032, 0.011429192, 0.023164717, -0.020101372, 0.023208478, -0.02148717, -0.00015168569, -0.031450335, 0.03719775, 0.007570836, -0.022172777, -0.019970087, 0.010641474, 0.023106368, -0.011567771, 0.01033514, -0.0064621978, 0.019780451, 0.01182305, 0.0053097964, -0.0024251475, -0.007979282, 0.0074359034, 0.029058008, -0.038277216, -0.02948104, 0.043295264, -0.019765863, -0.010721705, -0.009773526, -0.001685751, 0.034426153, -0.025396582, -0.014558178, -0.024127483, 0.016848393, -0.0006350057, -0.016527472, 0.010050686, -0.016629582, 0.00530615, 0.0024415583, -0.006345499, 0.0017568644, 0.048634235, -0.025294472, 0.003488201, 0.03515552, 0.014149733, 0.023514813, 0.011582359, -0.028766261, -0.0050654584, 0.0396776, -0.015127085, -0.027088715, 0.002496261, 0.027322114, -0.034630377, 0.007964695, -0.004595016, 0.03565149, 0.013332841, 0.0117136445, 0.028299466, 0.046329435, -0.016746283, -0.0448707, -0.05400238, 0.0015471712, 0.017315188, -0.0051019266, 0.0068779374, -0.0030305227, 0.017490236, -0.0014569119, 0.02088909, 0.010101742, -0.0019328244, 0.038656488, 0.063075714, -0.0057401233, -0.0023193893, -0.017957032, -0.0030706378, 0.024375467, -0.023223067, -0.018102905, -0.024827676, 0.021414235, -0.033696786, -0.0118595185, 0.0028919429, -0.008934754, 0.006312677, 0.02441923, -0.010189266, 0.0025436697, 0.015550119, -0.016862981, -0.021633046, 0.016804632, -0.01890521, 0.023325179, -0.0031526918, 0.008949341, 0.04711715, 0.019795038, -0.0027497162, -0.021516345, -0.02727835, -0.028270291, -0.0044272617, 0.0041902172, 0.020495232, 0.009226501, 0.038656488, -0.004350678, -0.00722074, 0.02805148, 0.034484502, -0.023471052, -0.005586956, 0.027365874, -0.015272959, 0.037577022, 0.06430105, -0.025936315, 0.014652996, -0.008416903, -0.02485685, -0.009926694, -0.009554717, 0.0397943, -0.05373981, -0.0015572, 0.03544727, 0.020422295, 0.0100725675, 0.026796969, -0.005484845, 0.0030396397, 0.020786978, 0.017183902, 0.012049153, 0.016687931, -0.042915992, 0.002106049, 0.022756271, 0.011181206, -0.011400017, 0.020218072, 0.010634181, -0.0072280336, 0.012027273, 0.03451368, -0.021647632, 0.02701578, -0.008934754, -0.00034371452, 0.01775281, 0.013150499, 0.003902117, -0.001729513, -0.060216594, 0.0072389743, 0.020509819, -0.019911738, -0.0050216964, 0.012231496, -0.011998098, -0.038014643, -0.014113264, -0.0028992365, -0.03285072, 0.020801567, -0.041019637, 0.002571021, -0.02761386, 0.0011214028, -0.0014496183, 0.012917101, 0.03075014, 0.0048101796, -0.028314052, -0.0029539391, -0.033200815, 0.03191713, 0.0048101796, 0.00821268, 0.04778817, -0.014470655, 0.014149733, -0.0022373353, 0.004288682, -0.017344363, -0.0051748636, -0.0023120956, -0.010028806, 0.002018525, 0.00079181977, 0.006341852, 0.010123624, -0.01719849, 0.023354353, 0.017446475, 0.014186202, -0.012494069, -0.0063965544, -0.03428028, 0.014390424, -0.010714411, -0.009642241, -0.005481198, 0.008730531, 0.0023357999, 0.0423325, -0.003923998, 0.009751646, -0.023558576, 0.008446078, 0.021005789, 0.01989715, 0.023354353, 0.009292144, 0.008723238, 0.0010667002, -0.03340504, -0.010284084, 0.020903677, -0.01874475, -0.015141672, 0.029364342, -0.03649756, -0.02755551, 0.01957623, -0.006998283, 0.014820751, 0.0068524093, 0.0110863885, -0.001020203, -0.026373936, 0.021005789, 0.0010712588, 0.0012490421, 0.011808462, 0.000775409, -0.0018580641, 0.0013949156, -0.001359359, -0.035972413, -0.018832274, -0.001117756, -0.03235475, 0.01339119, -0.031391986, 0.0025874318, -0.003732539, 0.02568833, -0.03185878, 0.009576597, -0.0025637273, 0.012858752, -0.027292939, 0.020013848, 0.029028833, 0.03693518, -0.037839595, 0.00010632813, 0.012362782, 0.015477182, -0.047758996, -0.0224937, -0.01918237, 0.018992733, 0.023514813, -0.008854523, 0.036176637, -0.009620359, -0.0058094133, -0.00050508714, -0.03915246, 0.01565223, -0.021968555, -0.0041683363, 0.028532863, -0.0129243955, 0.0029265878, 0.005280622, -0.025221534, -0.011538597, -0.008613832, -0.0032092177, -0.014171613, 0.02188103, 0.0036504853, 0.0075927177, 0.0058859973, -0.041077986, 0.0001701478, -0.040523667, 0.0010967866, 0.024069132, -0.0025673741, 0.008095982, 0.02326683, 0.023981608, -0.015841866, 0.04338279, 0.0024342646, 0.015404245, -0.0040953993, -0.025236122, 0.011575065, 0.072236575, -0.02535282, -0.030575093, 0.029962424, -0.0072936765, 0.019269893, 0.0062689153, 0.0012189557, -0.030458394, -0.00914627, -0.041428085, -0.004963347, -0.008205386, -0.014426893, 0.0039495258, -0.016308662, -0.0121512655, 0.0031435746, 0.05522772, 0.004605957, 0.018088318, 0.041165512, -0.030079123, -0.0008191711, 0.014893688, 0.031129412, 0.028284878, 0.015083323, 0.018861448, 0.021793505, -0.0009126213, -0.016833806, -0.02188103, -0.002065934, -0.03509717, 0.013340135, -0.008453371, 0.024054546, -0.012049153, -0.0035647845, -0.0039604665, 0.03130446, 0.020801567, -0.0005707302, 0.010028806, 0.024039958, 0.0036413681, 0.025381995, -0.005007109, -0.027263764, -0.026096776, 0.034688726, 0.0066700676, -0.019255307, 0.009620359, -0.040261097, -0.017840333, -0.020684866, 0.006272562, 0.0025418464, -0.009642241, 0.04303269, -0.024754738, -0.0063637327, 0.019809626, -0.036234986, -0.013960097, -0.011582359, -0.015695993, 0.011064508, -0.031537857, -0.039473377, 0.020086786, -0.009328612, 0.023660688, 0.0014888218, 0.011881399, -0.0010302318, 0.023952434, -0.022639573, -0.0070967474, 0.0040990463, -0.050501417, 0.0010037923, -0.00077176216, -0.007665654, 0.021341298, -0.023704449, -0.019196957, 0.017738221, -0.029218469, 0.019138608, 0.029991599, 0.051084913, 0.005411908, -0.010546656, 0.00015556045, -0.013135912, -0.009438017, -0.028795436, -0.011691764, -0.016731694, -0.013347428, -0.0009012249, -0.0077240034, -0.041573957, -0.046154387, 0.002118813, 0.021064138, 0.0067575914, 0.024667215, -0.028197354, -0.0037070112, -0.004981581, -0.020407706, -0.0060829264, -0.013084856, 0.027322114, -0.03247145, -0.029904075, 0.0074978997, -0.02188103, -0.012275258, 0.0007403082, -0.0026239003, 0.008898285, 0.010976983, 0.0161482, -0.013733993, 0.003767184, -0.017461061, 0.013821517, 0.012749347, -0.005597897, 0.01077276, 0.0063491454, 0.009168152, -0.029145531, -0.044287205, -0.00083011156, 0.024535928, 0.011348961, -0.0050836927, -0.013923629, 0.022960493, -0.02700119, 0.0026950135, 0.024375467, -0.0074760187, 0.024025371, -0.020772392, -0.024477579, 0.014689465, 0.0001236506, 0.0022956848, -0.039006583, -0.005893291, 0.0065497216, -0.024725564, -0.013048388, 0.018496765, -0.008584658, 0.005565075, 0.012523243, 0.031654555, 0.015973153, -0.012384663, 0.021034963, 0.025338233, 0.011553184, 0.019926324, -0.007454138, -0.02810983, 0.022566635, -0.014186202, 0.0026129596, -0.017461061, 0.030691791, 0.0029211175, 0.018978147, -0.005437436, 0.0032383925, -0.018876035, -0.033084117, -0.004536667, 0.02650522, -0.01083111, -0.024681803, -0.014536298, 0.015725167, 0.0028044188, -0.03915246, -0.006972755, -0.0003494127, 0.029904075, 0.056044612, -0.021370472, -0.003883883, -0.007694829, -0.0077750594, -0.017533999, -0.021560108, -0.0050836927, -0.00658619, -0.035855714, 0.017052617, -0.016002327, 0.013055681, 0.0041610426, 0.008147037, -0.017183902, -0.01339119, -0.016002327, 0.02805148, 0.004303269, 0.0061485693, 0.008504427, -0.0013520654, -0.007337439, -0.005346265, 0.009496367, -0.011093683, 0.018861448, -0.004040697, -0.009168152, 0.02656357, -0.007760472, -0.01515626, -0.0037635372, -0.007264502, 0.018978147, -0.03594324, -0.016191963, 0.03168373, 0.023996197, 0.00701287, -0.024929786, 0.019153195, 0.0037817713, 0.012289845, 0.0019565288, 0.037460323, -0.015054149, 0.019765863, -0.028197354, 0.025454931, -0.0020075846, -0.012690998, 0.0054447297, -0.013639175, 0.029831138, 0.0051858043, -0.043849584, -0.03781042, -0.013135912, -0.033113293, -0.0017668932, 3.333438e-05, -0.008358553, 0.021180836, -0.060274944, -0.0054994323, -0.00862842, 0.017738221, 0.03509717, 0.01884686, -0.014981212, -0.003975054, -0.028372403, 0.02166222, -0.022172777, 0.010225735, -0.027526336, 0.008059513, -0.0114729535, 0.03620581, 0.006637246, 0.001896356, -0.010801936, 0.0106998235, 0.01126873, -0.024652626, -0.016994266, 0.0045038452, 0.001940118, 0.01339119, -0.026432285, -0.0073921415, 0.036234986, -0.014594647, 0.02838699, 0.025381995, -0.019765863, -0.01565223, -0.005262388, 0.017986206, -0.0051821573, -0.021983141, -0.03515552, 0.015302134, -0.027978543, -0.02215819, -0.0022610398, -0.033871833, 0.014317487, 0.02705954, 0.02277086, 0.048400838, -0.035739016, -0.009379668, 0.017183902, -0.0080449255, -0.015535532, -0.009022278, -0.022741685, -0.015185435, -0.008234561, -0.007104041, -0.01408409, 0.010167385, -0.008460665, 0.03191713, 0.012880633, 0.005933406, -0.003953173, -0.005842235, -0.011210381, 0.0023248594, 0.0059005846, -0.0016046088, 0.012267964, 0.034426153, 0.038072992, -0.014448773, 0.021676807, -0.023485638, -0.019940913, 0.02331059, 0.010947809, -0.010138211, 0.004131868, -0.01940118, -0.03322999, -0.016717106, 0.0111374445, 0.052660346, -0.0005762005, -0.02094744, -0.024769327, 0.004015169, 0.0120053915, 0.004441849, -0.009087921, -0.037985466, 0.002640311, 0.0273367, 0.033434212, -0.0012298962, -0.009496367, -0.025498694, -0.043411963, -0.025717504, -0.030195821, -0.0052076853, -0.020407706, 0.0124867745, -0.042682596, -0.022522874, -0.0057474175, 0.01549177, -0.0055614286, 0.015287546, 0.033521738, -0.02751175, -0.025061073, -0.020101372, -0.02188103, -0.028562037, 0.034484502, -0.036234986, 0.024813088, -0.021020377, 0.013464127, -0.03235475, 0.022843795, -0.0012198674, -0.016381597, 0.01813208, -0.005459317], "content_hash": "1c3e110214a54a3579690f8364a21180"}, {"content": "# Experiment: Your First Ablation\n\n## Goal\n\nLearn how ablation works by removing an attention head and observing how it changes the model's prediction. Discover which heads matter and which are redundant.\n\n## Prerequisites\n\n- Complete \"Your First Analysis\"\n- Complete \"Exploring Attention Patterns\" (so you know about head categories)\n\n## Steps\n\n### Step 1: Set Up the Analysis\n\n1. Select **GPT-2 (124M)** and enter the prompt: `The cat sat on the`\n2. Set **Number of Generation Choices** to 1 and **Number of New Tokens** to 5.\n3. Click **Analyze**.\n4. Note the model's prediction in Stage 5 and the generated sequence.\n\n### Step 2: Select a Sequence for Comparison\n\n1. In the generated sequences section, click **\"Select for Comparison\"** on the generated sequence.\n2. This stores the original generation so the ablation experiment can compare against it.\n\n### Step 3: Find a Head to Ablate\n\n1. Expand **Stage 3 (Attention)** and look at the head categories.\n2. Find a head from the **Previous-Token** category. Let's say it's **L0-H3** (yours may differ).\n3. Note this head -- Previous-Token heads often have noticeable effects when removed.\n\n### Step 4: Set Up the Ablation\n\n1. Scroll down to the **Investigation Panel** and make sure the **\"Ablation\"** tab is selected.\n2. In the **Layer** dropdown, select the layer of your chosen head (e.g., 0).\n3. In the **Head** dropdown, select the head number (e.g., 3).\n4. Click the **+** button to add it. You should see a chip appear: \"L0-H3\".\n\n### Step 5: Run the Ablation\n\n1. Click **\"Run Ablation Experiment\"**.\n2. Wait for results to appear.\n\n### Step 6: Analyze the Results\n\nLook at the ablation results:\n\n- **Full Generation Comparison**: Compare the original text to the ablated text. Did the generated sequence change?\n- **Probability Change**: Look at the immediate next-token probability change. For example, \"72.3% \u2192 45.1% (-27.2%)\" would mean removing this head significantly reduced the model's confidence.", "source_file": "experiment_first_ablation.md", "chunk_index": 0, "embedding": [0.020743836, 0.030308522, 0.027893608, -0.0019095455, -0.030606994, -0.01440808, -0.016958663, 0.035436824, 0.04211175, 0.051120196, 0.057252448, -0.058229264, -0.04137914, -0.006352715, 0.025695765, 0.0066986717, 0.03372739, -0.011633643, 0.010717875, 0.07814552, -0.015384899, 0.007068371, 0.0044974373, 0.03513835, 0.020431798, -0.04273583, 0.049302224, 0.02302308, 0.030444192, -0.017040065, 0.0205539, -0.009055926, -0.053507973, -0.01569694, -0.028599089, -0.0044601285, -0.006657971, 0.013641548, -0.030172853, 0.007855252, 0.01043297, -0.05242262, -0.027038893, 0.039968178, -0.027568003, 0.051255863, 0.036793515, 0.0039920695, -0.022046262, 0.024013467, -0.0029101067, 0.06409018, -0.03372739, -0.0034086914, -0.01971275, 0.0012998815, -0.009571469, -0.0019705966, 0.014923624, -0.007543213, -0.010378702, 0.03351032, 0.049030885, 0.022588938, -0.014503049, -0.027269531, -0.039181296, 0.04208462, 0.01767771, 0.004083646, 0.021679955, -0.017040065, -0.0051757838, -0.0053114532, -0.02456971, -0.0428715, 0.002130008, -0.0069428766, -0.017596308, -0.016415985, 0.053860713, 0.027310232, 0.044173926, -0.011308037, 0.0013533013, -0.029982917, -0.057523783, -0.022697475, -0.0324521, -0.04813547, 0.002016385, 0.054213453, -0.032153625, 0.02143575, 0.007855252, 0.014245277, 0.005996583, -0.022995947, 0.019142939, 0.029982917, 0.027296664, 0.00548104, -0.021408616, -0.014964324, 0.022439703, -0.00565741, -0.029738711, 0.031176805, -0.0308512, -0.0125222765, -0.027459467, -0.018301789, 0.0052571855, 0.008628568, 0.012976769, 0.004565272, -0.019875553, 0.004551705, 0.019034404, -0.065881014, -0.036630712, -0.0023165534, 0.019251475, -0.009490068, -0.0024200012, -0.01657879, 0.0583378, -0.03470421, 0.02143575, -0.017433506, 0.04148767, 0.035626758, -0.018071152, -0.05888048, -0.020608168, 0.015846174, -0.017894782, 0.045720555, -0.069842555, 0.031936556, 0.010880678, -0.008838855, -0.023809964, 0.04349558, -0.07353276, -0.0043176757, 0.03535542, 0.068702936, -0.022249766, 0.033781655, 0.009910643, 0.030091451, 0.009429017, -0.020445364, -0.012054218, -0.049112286, -0.013126005, -0.011036698, 0.007997706, -0.07494372, 0.05768659, 0.011111316, -0.0038971007, -0.02006549, -0.033618852, 0.0583378, -0.040565122, 0.017637009, -0.019373577, -0.019739883, -0.028056413, -0.021978427, 0.0020587817, -0.025736468, -0.027866475, 0.022697475, -0.07803699, -0.015330631, -0.023633594, -0.051500067, -0.0005867697, -0.007373627, -0.0427901, 0.019346442, -0.03209936, -0.044228192, 0.014258844, -0.0056404513, -0.04281723, -0.037960272, 0.029223168, -0.002930457, 0.029331703, 0.0025488872, -0.038828555, -0.0028354884, 0.06560968, 0.028490555, 0.040266648, 0.00820121, 0.030389924, 0.020703135, 0.013187056, 0.027554436, 0.036440775, 0.018261088, -0.013722951, 0.019984089, -0.034405734, -0.027473034, 0.03470421, -0.07374983, 0.023063783, -0.0018484943, -0.047972664, -0.02153072, -0.040917862, 0.037010588, 0.047918398, 0.03907276, 0.031176805, -0.017542042, 0.032669168, 0.08085891, -0.0071633393, 0.009605386, 0.050930258, -0.024800349, 0.019821286, -0.03372739, 0.003995461, 0.021272946, -0.021666387, 0.011016347, 0.00069996883, -0.08606861, 0.032343563, 0.015819041, 0.015344198, -0.015724072, 0.008852422, 0.006746156, 0.03323898, -0.013743301, 0.03646791, 0.015968278, -0.02117798, -0.09209232, 0.014326679, 0.055678684, 0.02938597, 0.009951344, -0.04848821, 0.008479332, -0.017270703, -0.009585036, -0.008364012, -0.007760284, -0.017284269, -0.019617781, -0.019210773, -0.006837733, -0.025763601, 0.018953001, -0.021286514, -0.012386608, -0.008018056, 0.0022690692, -0.026808254, -0.02117798, -0.041704744, -0.0118574975, 0.019387143, 0.015452734, -0.04582909, 0.010439753, -0.0016890828, -0.01219667, -0.018505294, -0.0056608017, -0.011403005, -0.018953001, 0.031990822, -0.002599763, -0.031095404, 0.01604968, -0.017745545, -0.0043312428, -0.0037478646, -0.004901054, 0.030796932, -0.042274557, 0.07440104, -0.019102238, -0.019604215, 0.012481576, -0.0069360933, 0.018668097, -0.042274557, 0.035653893, 0.027947877, -0.008872773, 0.054376256, -0.017704844, 0.030661263, 0.029575909, 0.013594064, -0.00058253005, -0.004388902, 0.037770335, 0.012990336, -0.011403005, 0.0037648233, -0.022046262, -0.024651112, -0.0066986717, 0.010778926, -0.009978477, -0.030796932, -0.005046898, -0.028626224, -0.08009916, 0.026021373, 0.0016000499, 0.000405736, 0.018505294, 0.00042460253, 0.005694719, 0.021544285, -0.00097597105, 0.054891802, -0.014774388, -0.008642134, -0.010392269, -0.029196035, 0.0205132, -0.039588302, 0.06099692, 0.018491726, 0.048216872, -0.04618183, 0.01587331, -0.019522812, 0.007387194, -0.046697374, 0.056329895, 0.0066308375, -0.02797501, 0.040348053, -0.0064239413, -0.05025191, 0.013105655, -0.025695765, -0.02364716, 0.037146255, 0.053318035, -0.0025980673, -0.03028139, 0.032967642, -0.022222633, -0.008146942, 0.049003754, 0.017691277, 0.004019203, 0.019373577, 0.00925943, -0.018708797, 0.024094868, 0.008852422, -0.006172953, 0.0062848805, -0.03204509, 0.0018891951, -0.023402955, -0.06788892, -0.016470253, -0.009130544, -0.010168415, -0.015520568, 0.046507437, 0.0004464368, -0.007936655, -0.038692884, -0.0669121, 0.04230169, 0.021964861, -0.025275191, 0.012250938, -0.063221894, 0.024773216, 0.028843295, -0.017026497, -0.027744373, 0.009978477, -0.026102774, 0.06892001, -0.025627932, 0.0530467, 0.037906002, 0.035246886, -0.00017965584, -0.0015356069, -0.03239783, -0.042844366, 0.025695765, 0.0058066463, 0.0141774425, 0.0151135605, 0.023877798, 0.038123075, 0.059803028, 0.008458981, 0.037309058, 0.0011633643, 0.01569694, -0.0006575722, 0.03359172, 0.04184041, 0.007434678, -0.022127664, -0.0291689, -0.061756667, -0.009707139, -0.04265443, -0.035463955, -0.009917426, -0.0038021323, 0.00908306, 0.03478561, -0.06094265, -0.02135435, 0.03204509, 0.030579861, -0.029820113, -0.035898097, 0.009890292, -0.0026319844, 0.084006436, -0.03386306, -0.012590111, 0.023362255, 0.007054804, -0.0016178564, 0.0020706528, -0.051554337, 0.005111341, -0.03717339, -0.08004489, -0.0049383626, -0.03720052, -0.024895318, -0.021652821, 0.0019231124, 0.041976083, 0.027459467, -0.057578053, 0.04699585, -0.01489649, 0.10072089, -0.0055827918, 0.03147528, -0.041134935, -0.018301789, 0.041352004, -0.024908883, 0.037580397, 0.00978854, -0.027337365, 0.021028742, -0.024556143, 0.030824065, -0.009469718, -0.00068089034, -0.026862523, -0.0376618, -0.0145708835, 0.018925868, 0.022331167, 0.001865453, -0.036359373, 0.0101412805, -0.009836025, 0.051418666, 0.028409153, -0.012345906, 0.05812073, -0.01900727, -0.008472548, 0.015208529, 0.00051427144, -0.00017181245, -0.0023826922, 0.018939435, -0.015289931, 0.035653893, -0.032506365, -0.0126715135, 0.0043651597, 0.00822156, 0.0009166157, 0.046833042, -0.017596308, 0.020160459, 0.03416153, -0.005531916, -0.01900727, 0.03513835, 0.030335657, -0.020486064, 0.06501273, -0.035192616, -0.019902688, -0.005124908, -0.0011557329, -0.010059879, 0.020743836, 0.021313647, 0.006973402, 0.017528474, -0.00082037534, -0.016483821, -0.0010124322, -0.017637009, 0.017094333, 0.00978854, -0.0014762516, -0.034649942, -0.028870428, -0.03779747, -0.0011277511, 0.026401246, -0.020676002, -0.006980186, 0.030308522, -0.038611483, -0.012630812, 0.013553363, 0.0052537937, -0.060834117, 0.001902762, 0.003310331, 0.0046195397, 0.034025863, -0.019604215, 0.021327214, -0.0063696736, 0.055787217, -0.012278072, 0.022453269, 0.008872773, -0.003564711, -0.025736468, 0.01507286, -0.017243568, -0.021069443, 0.0036257622, -0.029331703, -0.04322424, 0.003476526, -0.037607532, 0.0072379573, -0.04281723, -0.012542627, -0.027920743, -0.01286145, 0.02161212, -0.006403591, -0.021815624, -0.0023877798, -0.011348737, 0.011409788, -0.014123174, 0.02638768, -0.03682065, 0.032289293, 0.016958663, -0.019332876, 0.010670391, 0.009462934, -0.007156556, -0.016999364, -0.0064578587, -0.04148767, 0.04173188, -0.011592942, -0.04905802, 0.013533013, 0.0020180808, -0.04070079, -0.005457298, 0.036250837, -0.03209936, 0.03247923, 0.04173188, -0.034921277, 0.0032391048, 0.027703673, 0.0070819375, 0.0062713134, 0.0032882849, 0.029575909, 0.012630812, -0.0053351955, 0.049302224, -0.012223804, 0.01337021, -0.036603577, -0.0029084107, 0.021096576, 0.008709969, -0.0006376457, -0.006895392, -0.042898633, -0.008146942, -0.016605923, 0.028002145, -0.00035825177, -0.0031068271, 0.01630745, -0.020486064, 0.021015175, -0.009157678, -0.0025726291, 0.001026847, 0.024610411, 0.018057585, -0.016822994, -0.0054335557, 0.022426136, 0.0063628904, -0.044987943, 0.039452635, 0.034731343, 0.014977891, -0.008194426, 0.015439167, 0.0034578715, -0.0004731467, 0.015629103, 0.002094395, -0.011715044, -0.017460639, 0.011036698, 0.001420288, -0.016117513, 0.015927576, 0.014014639, -0.0010310867, -0.029901514, -0.030824065, -0.016483821, -0.020472499, 0.07450958, 0.013227757, 0.017284269, 0.002801571, -0.0019485503, -0.039968178, 0.00720404, -0.0076788827, -0.037607532, 0.047945533, -0.013302376, -0.007943437, 0.009137328, 0.0030763014, -0.0010828107, 0.007041237, 0.004388902, -0.0039683273, -0.044662334, -0.0070955046, 0.010521155, -0.008526816, -0.0069089592, 0.03853008, -0.0038564, -0.02930457, -0.0070073195, 0.024244105, -0.006003367, 0.017650576, -0.033320382, 0.021055875, -0.014136742, -0.017976183, 0.00317127, -0.033347514, 0.0006397656, -0.00057786645, 0.005216485, -0.03747186, -0.009490068, -0.011626859, -0.007387194, 0.020825239, -0.029738711, 0.031448144, -0.010677175, -0.101155035, -0.02550583, -0.013987505, 0.036305107, 0.022602506, -0.017433506, 0.00624418, 0.014747254, 0.06143106, 0.016388852, -0.024854617, -0.03028139, -0.0029982915, 0.009876725, 0.034622807, -0.035952367, 0.038665753, -0.022711042, -0.0034086914, -0.013512663, -0.047267187, -0.049709234, 0.032723438, -0.018233955, 0.008669268, 0.026618317, -0.030905468, 0.004148089, -0.010066662, -0.008947391, 0.012067785, 0.029548775, 0.010867111, 0.018261088, -0.040727925, 0.014435214, 0.019794151, -0.013845053, -0.014109608, 0.0017789637, -0.025614364, 0.022561805, 0.017297836, 0.033971593, 0.031231074, 0.0032357129, 0.005986408, -0.0050028055, 0.01248836, -0.040293783, -0.015981844, -0.020676002, 0.012122053, -0.008791371, -0.003690205, 0.019861987, 0.031258207, -0.005796471, -0.019984089, -0.012081351, 0.019780584, 0.0037817818, -0.029277436, 0.016321016, 0.014692985, -0.034052994, -0.02002479, 0.03280484, 0.008323312, 0.020458931, 0.0360609, 0.0068614753, 0.012210238, 0.033917326, 0.008438631, 0.010290517, -0.023294419, 0.049085155, -0.011660777, -0.004572056, 0.01622605, 0.0060712015, 0.0044431696, -0.040592257, -0.0019163289, -0.0066037034, -0.017460639, -0.028002145, 0.004890878, -0.0378246, -0.029223168, 0.010507587, 0.010690741, 0.022860277, -0.013451612, 0.0103515675, 0.016063245, -0.01182358, -0.10142637, 0.0031848368, 0.0014601409, -0.0069225263, -0.030932602, -0.011470839, 0.0085743, 0.012983552, 0.008146942, -0.0094358, 0.027215263, -0.03888282, 0.003476526, -0.008872773, 0.030796932, -0.027147427, 0.025967104, -0.0205946, 0.027120294, -0.0205946, -0.018627396, 0.01630745, -0.015507001, -0.012474792, -0.044906538, 0.014191009, -0.023525057, 0.011911765, 0.018193254, 0.041569076, -0.01971275, 0.018640961, -0.035002682, 0.017162167, -0.0132752415, 0.032642033, -0.018301789, 0.026129909, 0.02617061, 0.006434117, 0.027947877, -0.007115855, 0.008438631, -0.005952491, 0.03288624, -0.011403005, -0.016239615, 0.0070887213, 0.0023945633, 0.00642055, 0.010331217, 0.004134522, 0.01957708, 0.01458445, 0.025478695, -0.027120294, -0.0009717314, -0.026930356, 0.018179687, 0.03394446, 0.017352104, 0.053290904, 0.03147528, -0.0102023315, -0.016497387, 0.017202867, -0.010066662, 0.040510856, -0.044445265, -0.05244975, -0.035545357, -0.009089843, -0.014801521, 0.00083394226, -0.0052673607, 0.00393441, 0.014665851, 0.049112286, -0.009110194, -0.012990336, 0.025424428, -0.014557316, -0.017514907, 0.0011107924, -0.0032815014, 0.03725479, 0.0026235052, 0.0035206184, 0.007054804, 0.056764036, 0.026998192, -0.02368786, 0.016334584, 0.021910593, -0.04645317, -0.022751743, 0.024081301, 0.0075906976, 0.032316428, 0.011884631, -0.006980186, 0.0019875553, -0.013112438, -0.013627982, -0.03646791, 0.0063493233, 0.016280316, -0.007984139, -0.016239615, -0.015805474, 0.04132487, 0.024189837, -0.017881215, 0.026713286, -0.016985796, 0.024556143, 0.054023515, 0.0075906976, -0.017406372, -0.017121466, -0.007244741, 0.03142101, -0.010344785, 0.00040001248, 0.010222682, 0.01630745, -0.03823161, -0.007136205, -0.0066172704, 0.017216435, 0.027785074, 0.009646088, -0.025261624, -0.014394513, 0.0012193279, -0.0064510754, -0.013302376, 0.0033205063, -0.01524923, 0.0017060415, -0.0360609, -0.009042359, 0.017772678, 0.0050163725, -0.051852807, -0.0222769, 0.0022724608, -0.017772678, -0.0079095205, 0.022711042, 0.0117082605, 0.029765846, 0.0010319346, 0.011477623, 0.0005973689, -0.008838855, 0.023090916, 0.0063662822, -0.025912838, 0.032235026, 0.015086426, 0.0066952803, 0.034107264, 0.00446352, 0.0020214727, -0.00090559263, -0.029982917, -0.0062204376, -0.01062969, -0.0119185485, -0.026496215, 0.04341418, 0.015018592, 0.041921817, 0.03112254, 0.019685617, -0.019115806, -0.017175734, -0.014923624, 0.020174026, 0.018573128, 0.010317651, -0.0052470104, -0.023674294, -0.018233955, -0.021001609, 0.009781756, 0.0074007604, 0.010995997, 0.001282075, -0.008458981, 0.02617061, -0.02452901, 0.007726367, -0.011904982, -0.009035576, -0.004500829, 0.017786246, -0.049736366, -0.039913908, -0.042573027, 0.054810397, 0.01609038, -0.017311404, 0.052232683, 0.039045624, -0.019346442, 0.023294419, -0.029250301, -0.006013542, -0.01595471, 0.009734273, -0.017569175, 0.0040768627, -0.029847248, 0.0073261424, -0.018545995, 0.00012591807, 0.015289931, -0.019129371, -0.0754864, -0.036874916, -0.030254254, 0.014665851, 0.0016704283, 0.018410325, 0.031041136, -0.009666438, 0.009252647, 0.005135083, -0.03416153, -0.02382353, -0.022792444, -0.0042566243, -0.03353745, 0.038421545, -0.0031560073, -0.0075160796, 0.0054064216, 0.009171245, 0.049736366, -0.00802484, -0.012875017, -0.018220387, 0.009673221, -0.014503049, 0.039913908, 0.018803766, 0.013105655, -0.022154797, -0.017365672, -0.0031984039, 0.0041446975, -0.0011277511, 0.018464593, 0.0011921941, 0.034649942, 0.008764237, 0.05101166, -0.00736006, -0.022928113, 0.032913372, 0.0016848432, -0.05139153, 0.0026726853, -0.017731978, -0.023036648, -0.014733686, 0.024081301, 0.002715082, 0.03801454, 0.015099994, -0.014326679, 0.019563513, 0.0075703473, -0.021829192, 0.013824702, -0.014001072, 0.021205112, -0.015167829, 0.017718412, 0.037363328, -0.015357765, 0.002247023, -0.0056608017, -0.019767018, -0.020608168, -0.005850739, -0.007041237, -0.012284855, -0.013906104, 0.013451612, 0.009062709, -0.017175734, -0.007387194, 0.007977355, 0.0050265477, -0.0029711577, 0.0036359373, -0.028409153, 0.0061966954, -0.012936068, 0.017501341, -0.034568537, -0.015099994, 0.019739883, 0.0151135605, -0.063981645, -0.02607564, 0.01573764, -0.015995411, 0.022304034, 0.008832072, 0.0077195833, -0.007672099, -0.025465129, 0.0042667994, -0.021367915, 0.0038903174, -0.033781655, -0.036630712, 0.023063783, -0.019780584, 0.033130445, -0.011986383, -0.02337582, -0.010439753, -0.01096208, -0.0068580834, -0.0119795995, 0.0033340731, 0.010819627, -0.010093796, 0.0102023315, 0.0013617807, 0.019943388, -0.019658482, 0.0012286551, 0.027690105, 0.0047484254, -0.027378066, -0.00069869694, -0.040429454, 0.011145233, -0.001826448, -0.004327851, -0.005633668, -0.02413557, 0.019644916, 0.03893709, 0.08232413, -0.05502747, -0.038123075, -0.025695765, 0.012264505, -0.00784847, -0.009557903, -0.01926504, -0.045720555, -0.008350446, -0.02811068, 0.015832609, 0.0012015213, 0.021367915, -0.0027846126, 0.012386608, -0.03470421, 0.008296178, 0.0393441, 0.017202867, 0.0077535007, 0.064741395, -0.039968178, 0.001686539, 0.031610947, 0.011179151, 0.023538625, 0.025315892, -0.005921965, 0.018858034, -0.0073329257, -0.041623343, -0.011172367, -0.013159922, -0.026645452, 0.019916253, 0.02881616, 0.0077399337, -0.02687609, 0.0018111853, -0.018939435, 0.034052994, 0.0072718747, -0.010880678, 0.007780635, 0.008689619, -0.008547166, 0.006434117, 0.0077060163, -0.0064646425, -0.012013517, -0.0074753785, -0.0027303447, -0.0132752415, 0.039669704, -0.024108436, -0.0024233928, 0.018790198, 0.017691277, -0.007000536, -0.015547702, 0.038313013, -0.017745545, 0.009754623, -0.015371332, -0.03155668, -0.03302191, 0.011592942, -0.0015627408, 0.016958663, -0.028164947, -0.012976769, 0.021720655, -0.011504757, 0.017257135, 0.008960958, -0.027568003, -0.023090916, 0.039723974, 0.0114369225, -0.0070887213, -0.033211846, -0.028409153, -0.021639254, -0.010012395, -0.0020655652, 0.028843295, -0.0046195397, -0.041948948, 0.008933824, -0.012983552, 0.025139522, 0.049709234, 0.029250301, 0.027079593, -0.010636473, -0.028626224, -0.034324333, 0.012101702, -0.016822994, 0.017949048, 0.004782343, -0.012576545, 0.00822156, -0.029982917, 0.0054844315, -0.031529546, 0.02174779, 0.014692985, 0.0050842073, 0.015086426, -0.00626453, -0.015859742, -0.0038530082, 0.0059830165, 0.021164412, -0.0009819065, 0.05011624, -0.033266112, -0.014204577, 0.01842389, -0.010907812, 0.0012583327, -0.0034019079, 8.081863e-05, -0.012440875, 0.03220789, 0.004578839, -0.003400212, -0.0005168152, 0.003169574, -0.007672099, 0.018396758, -0.03120394, 0.0070073195, 0.0010259991, -0.021028742, 0.0004744186, -0.05909755, 0.04181328, -0.011443706, 0.011009564, -0.01476082, -0.04175901, -0.0061085103, 0.02550583, 0.006027109, -0.0040531205, 0.011335171, -0.023701428, -0.024447609, -0.00064697297, 0.010704308, 0.00788917, 0.0016763639, -0.04004958, -0.010005611, 0.013261675, 0.010758576, 0.0222769, 0.020268993, 0.0009963214, -0.01988912, 0.0068512997, -0.0031424402, 0.015317065, -0.032343563, 0.010480454, 0.053887848, 0.016008979, 0.0018841075, 0.002411522, -0.00061220775, -0.0010370222, -0.00094035786, -0.00927978, 0.01427241, 0.006579961, 0.002715082, 0.019129371, -0.015588403, -0.0052436185, -0.017012931, -0.02153072, -0.012359474, 0.031149672, -0.021856325, -0.0031678784, -0.008581083, 0.0059796246, 0.008709969, -0.031041136, 0.0021401832, 0.0017077373, 0.030308522, 0.01047367, 0.00930013, 0.0073532765, 0.02010619, 0.0011481015, -0.00017287237, -0.014977891, -0.031231074, -0.026835388, -0.028300617, 0.023240153, 0.015615537, -0.02010619, -0.0067665065, 0.009761406, -0.019984089, -0.00401242, -0.026753986, -0.017894782, 0.0037207308, 0.008323312, 0.01595471, 0.002004514, -0.0023402956, -0.011504757, -0.018749498, 0.009347615, 0.053209502, 0.0021893634, -0.02895183, 0.007611048, -0.020703135, -0.034460004, 0.0013940021, 0.0066681462, 0.0010463495, -0.021584986, -0.011681127, 0.0066918884, -0.006966619, 0.017352104, -0.012617245, 8.823804e-05, 0.00017541616, 0.0053487625, 0.025288759, 0.030172853, 0.025166657, 0.007773851, 0.0049553216, 0.0427901, -0.0013516054, -0.034921277, -0.00892704, -0.033971593, 0.03424293, 0.016687324, -0.023972766, -0.004182006, -0.023497924, -0.004093821, 0.0035714945, 0.010405836, -0.012325556, 0.021584986, -0.047321454, -0.0093272645, -0.018844467, -0.0029338486, 0.010941729, 0.032642033, -0.0061831283, 0.0026964275, -0.03690205, 0.02938597, -0.023457224, -0.0222769, -0.039127026, -0.01613108, 0.019631349, 0.04243736, 0.013051387, 0.0072379573, -0.01060934, -0.01182358, 0.02435264, -0.03169235, -0.002891452, -0.011348737, -0.0017060415, 0.009646088, 0.019631349, 0.0057320283, 0.011660777, 0.0058778725, 0.010154847, 0.0308512, -0.0037987405, 0.011348737, 0.014882923, -0.00036948686, -0.009381533, -0.0077399337, -0.006820774, 0.0062747053, -0.036712114, -0.020011222, 0.017542042, -0.002816834, 0.011742178, 0.013044603, 0.006925918, 0.038991358, -0.01918364, -0.01235269, -0.028029278, -0.016334584, -0.017419938, -0.0055081737, 0.03345605, 0.025573663, 0.016497387, -0.027418766, -0.03451427, 0.0019349834, 0.022778876, 0.021951294, -0.0043787267, -0.0010700916, 0.013987505, -0.011579375, -0.003578278, 0.025044553, 0.024868183, -0.014435214, 0.0037783901, 0.029982917, 0.038421545, 0.029250301, -0.010975647, 0.0012473096, -0.008038406, -0.010548288, 0.031610947, -0.007611048, -0.013071737, -0.014123174, 0.0031526154, 0.02010619, 0.007509296, 0.0427087, 0.00052868633, 0.015493435, -0.01354658, 0.016809426, 0.008967741, 0.0069598355, 0.011016347, -0.015167829, -0.0012142403, -0.014855789, -0.012454442, 0.0014737078, 0.017135033, -0.040917862, -0.0427901, 0.014557316, -0.001104009, 0.0040531205, -0.027473034, 0.0024369599, -0.01816612, -0.029575909, -0.03247923, -0.009178028, -0.0060440674, -0.00022809402, 0.031068271, -0.0080316225, -0.030199988, -0.021693522, 0.0002206746, -0.03823161, 0.025071688, -0.016456686, 0.012936068, -0.026360545, 0.020743836, -0.013648332, 0.020716703, -0.012908935, -0.008730319, 0.02691679, 0.0008954174], "content_hash": "d22896e18178cfa41fb2ca3bb0a60bce"}, {"content": "the Results Look at the ablation results: - **Full Generation Comparison**: Compare the original text to the ablated text. Did the generated sequence change? - **Probability Change**: Look at the immediate next-token probability change. For example, \"72.3% \u2192 45.1% (-27.2%)\" would mean removing this head significantly reduced the model's confidence.\n\n### Step 7: Try Ablating a Different Head\n\n1. Click **\"Clear Selected Heads\"** to reset.\n2. Now pick a head from the **\"Other\"** category (these often have less obvious roles).\n3. Add it and run the ablation again.\n4. Compare: was the effect larger or smaller than the Previous-Token head?\n\n### Step 8: Compare Your Results\n\n| Head | Category | Probability Change | Generation Changed? |\n|------|----------|-------------------|-------------------|\n| L0-H3 | Previous-Token | (fill in) | (yes/no) |\n| L?-H? | Other | (fill in) | (yes/no) |\n\n**Typical findings**:\n- Previous-Token heads in early layers often cause noticeable probability drops when ablated\n- Many \"Other\" heads have minimal impact for simple prompts\n- The same head may matter more or less depending on the specific prompt\n\n## What You Should Learn\n\n- Ablation is a tool for measuring the importance of individual model components\n- Not all heads are equally important -- some are redundant\n- The effect of ablation depends on the specific input prompt\n- This technique is used by researchers to understand how models work internally\n\n## What's Next?\n\nMove on to **Experiment: Token Attribution** to learn a different approach -- instead of removing components, measure which input tokens drive the prediction.", "source_file": "experiment_first_ablation.md", "chunk_index": 1, "embedding": [0.022489404, 0.0122117605, 0.0347294, 0.00634588, -0.016376464, -0.0047858804, -0.007334115, 0.035407044, 0.019778816, 0.043115277, 0.04094116, -0.042155277, -0.061609387, -0.0028094107, 0.015529406, -0.006931762, 0.033261165, -0.015698818, 0.0020576464, 0.046672925, -0.024988227, -0.0027811755, 0.010962349, 0.048931748, 0.03921881, -0.031397637, 0.013665877, 0.01609411, 0.0390494, -0.012945877, -0.006730586, -0.018155288, -0.021162346, -0.0025852933, -0.03506822, 0.0030811753, -0.006017645, -0.0065717623, -0.015049406, -0.0168847, 0.03204705, -0.03741175, -0.041562337, 0.04447057, -0.03179293, 0.05615998, 0.03670587, -0.031143518, -0.030663518, 0.036847044, -0.0062894095, 0.07098351, -0.009352937, -0.010221173, -0.037242338, -0.009719997, -0.011971761, -0.018607052, 0.007814115, -0.02807999, -0.02567999, 0.0009494114, 0.042352926, 0.02741646, -0.002567646, -0.021656463, -0.036197633, 0.0087247025, 0.0042952923, 0.0056470567, 0.02855999, -0.016870582, -0.0018388228, 0.0044929394, -0.04077175, -0.048056453, 0.024409402, 0.004076469, -0.026414108, -0.01394823, 0.061439976, 0.013884701, 0.052856453, -0.02519999, -0.0040199985, -0.05375998, -0.036479987, -0.017745877, -0.03543528, -0.08244703, -0.036084693, 0.018762346, -0.018578816, 0.016941171, -0.024649402, 0.026682343, -0.0030264694, 7.63235e-05, 0.022489404, 0.005011763, 0.0176047, -0.002059411, -0.023421168, -0.019863522, 0.013687054, -0.017138816, -0.04867763, 0.0059294095, -0.047802337, 0.012303525, -0.034334105, -0.013319995, -0.042014103, 0.023251757, -0.007496468, -0.028094107, -0.032075282, 0.007955291, 0.039585866, -0.05782586, -0.024282344, -0.020512933, -0.016235288, 0.014654112, 0.0029029401, -0.01215529, 0.04111057, -0.039247043, -0.0016897053, -0.008301173, 0.04478116, 0.031510577, -0.021359991, -0.018776463, -0.03975528, 0.0040835277, 0.005347057, 0.036621165, -0.06640939, 0.04063057, 0.020978816, -0.034701165, -0.027035285, 0.010792937, -0.07764703, -0.00046411747, 0.041505866, 0.040404692, -0.027670577, 0.042155277, 0.014682348, -0.0028799989, -0.019891758, -0.00969882, -0.023378816, -0.060988214, -0.025214108, 0.016432935, -0.0140894065, -0.05963292, 0.02532705, 0.027670577, -0.033910576, -0.019976463, -0.019016463, 0.043143515, -0.014738818, -0.003688234, -0.037552927, -0.026682343, -0.051896453, -0.04017881, -0.005488233, -0.03831528, -0.038992926, 0.022348227, -0.07403292, -0.016009405, -0.01191529, -0.040122338, -0.01017882, 0.02377411, -0.030465871, 0.009261173, -0.020922346, -0.024465874, -0.01939764, -0.00054441154, -0.027769402, -0.024084697, 0.019482346, 0.020498816, 0.051896453, 0.014498818, -0.019807052, -0.013171759, 0.07002351, 0.007941173, 0.013884701, 0.010171761, 0.02551058, 0.014167054, 0.030719988, 0.012847054, 0.026414108, 0.006017645, -0.013411759, 0.012162348, -0.029082343, -0.028447049, 0.02987293, -0.062287036, 0.018310582, 0.0035399988, -0.054465864, -0.014597641, -0.032301165, 0.051444687, 0.016757641, 0.027007049, 0.0356894, -0.011668231, 0.03783528, 0.044922337, -0.010214114, 0.022178816, 0.050625864, -0.010482349, 0.017759994, -0.034221165, 0.019185875, 0.028009402, -0.011661172, -0.0095505845, -0.021543521, -0.08035762, 0.020244699, -0.012889407, 0.032131754, -0.02567999, -0.0035399988, -0.0059470567, 0.052376453, 0.008371762, 0.014682348, 0.049411748, -0.029251754, -0.05663998, 0.02783999, 0.05872939, 0.057515275, -0.013044701, -0.031651754, 0.0019923523, 0.0012070584, -0.008992938, -0.006494115, 0.003818822, -0.018465875, -0.007814115, -0.020484699, -0.017505877, -0.046729393, 0.02719058, -0.040574104, -0.014039995, -0.01729411, 0.015416465, -0.04593881, -0.042945866, -0.047858804, -0.019185875, 0.018861169, 0.004034116, -0.039952926, -0.022757638, -0.01358823, -0.002705293, -0.031482343, 0.014223524, -0.0003611028, -0.008703526, 0.046192925, -0.010870584, -0.024734108, 0.007517644, -0.0049305866, -0.00065823505, 0.009367055, -0.011131761, 0.01083529, -0.030945871, 0.071435265, -0.02041411, -0.035604693, 0.005537645, 0.009910584, 0.010482349, -0.04847998, 0.0137294065, 0.026089402, -0.03981175, 0.040602338, 0.0038117634, 0.02329411, 0.0036741162, 0.02551058, 0.020244699, -0.02035764, 0.009007055, 0.009705879, 0.025397638, -0.008124703, -0.020343522, -0.044865865, -0.011237643, 0.027910577, -0.03314822, 0.008823526, -0.02717646, -0.021924699, -0.049355276, 0.013284701, -0.013496466, -0.0016985288, 0.027303519, -0.010679996, -0.0129176425, -0.00044051453, -0.012430584, 0.03537881, 0.017590582, 0.015585877, 0.020343522, -0.03628234, 0.014781171, -0.057487037, 0.04430116, 0.022757638, 0.04147763, -0.042014103, -0.011223526, 0.008096468, 0.0047152922, -0.004108234, 0.04593881, 0.0064588212, -0.0069776443, 0.027204696, 0.020879993, -0.029251754, 0.0082588205, -0.0044258805, -0.04639057, 0.042804692, 0.080075264, -0.006797645, -0.040517632, 0.051896453, -0.004274116, -0.018395288, 0.036367044, 0.004521175, 0.01334823, 0.015797641, -0.0028729402, -0.035604693, 0.027402343, 0.0012397055, 0.00856235, -0.015938818, -0.011950584, -0.0050011748, 0.006314115, -0.09012703, -0.0075388206, -0.020639993, -0.0051211747, -0.024225874, 0.04514822, 0.026329402, -0.026202343, -0.027444696, -0.045232926, 0.03543528, 0.0056294096, -0.0020629405, -0.0052411747, -0.05615998, 0.022771757, 0.036254104, -0.0150070535, -0.0159247, 0.0029858812, -0.008265879, 0.07476703, -0.042494103, 0.06612703, 0.027515285, 0.044385865, 0.018183522, 0.03035293, -0.056668214, -0.022644697, 0.033374105, 0.019609405, 0.011428231, 0.009910584, 0.03077646, 0.035830576, 0.072225854, 0.009268232, 0.044075277, -0.017251758, 0.030381165, 0.0023629402, 0.039952926, 0.04190116, 0.021529404, -0.005788233, -0.03221646, -0.068385854, -0.008103526, -0.05246116, -0.0065470566, -0.010376466, 0.0050258804, 0.01191529, 0.012112937, -0.060141154, -0.03382587, 0.026583519, 0.0313694, -0.029054107, -0.01418823, -0.041759986, 0.0009970585, 0.072395265, -0.025214108, -0.016037641, 0.01275529, -0.00933882, 0.0065929387, -0.017011758, -0.06861174, 0.01257176, -0.028150577, -0.07476703, -0.004125881, -0.052828215, -0.040743515, -0.020428227, -0.02436705, 0.057317626, 0.0027158814, -0.047407042, 0.07346821, -0.0058164685, 0.09475761, -0.0060035274, 0.035887044, -0.04986351, 0.0059858803, 0.036254104, -0.0055235275, 0.049411748, 0.0150070535, -0.034644693, 0.024508227, -0.007997644, 0.012472937, -0.00018871317, -0.008428232, -0.028207049, -0.01999058, -0.01143529, 0.025821166, 0.0021052933, 0.0062470566, -0.053449392, 0.0043305866, -0.016221171, 0.048056453, 0.02519999, -0.040912926, 0.083858795, -0.0019605875, -0.005890586, 0.001951764, -0.008943526, 0.014597641, -0.024451755, 0.016997641, -0.016602347, 0.053505864, -0.005661174, 0.017675288, 0.014738818, 0.029647049, -0.0062788213, 0.053647038, -0.033684693, -0.0077858795, 0.05423998, -0.005735292, 0.00082544086, 0.0636988, 0.03410822, -0.024592932, 0.065279976, -0.03557646, -0.032611754, -0.011421172, -0.0056470567, -0.026498813, 0.0029699989, -0.005957645, 0.020004699, 0.012959995, 0.013094113, -0.013722348, 0.0190447, -0.007447056, 0.010559996, 0.0020911756, -0.012783525, -0.03681881, -0.007919997, -0.010023526, -0.0039423513, 0.03645175, -0.033035282, 0.00042330867, 0.039190575, -0.029675283, -0.02807999, 0.023265874, 0.0056858803, -0.043256454, 0.0035170575, 0.017082347, 0.022122344, 0.0031341165, -0.0095364675, 0.012924701, 0.0033776457, 0.050682336, -0.030465871, 0.016037641, -0.0030458812, -0.007990586, -0.021345874, 0.02759999, -0.029731754, -0.005717645, -0.0037941162, -0.017745877, -0.009239997, -0.01538823, -0.028475285, 0.027515285, -0.027402343, -0.009239997, -0.0004173528, -0.022404697, 0.030211754, 0.01729411, 0.012825877, 0.017195288, 0.0050329394, 0.007404703, -0.017887052, 0.032724693, 0.005064704, 0.022305874, 0.0024229402, 0.020992933, 0.00010130511, 0.019764699, 0.0051741158, 4.701285e-06, -0.03653646, -0.016898816, 0.03083293, -0.017985877, -0.034051754, 0.017971758, 0.017279994, -0.042352926, -0.0050505865, 0.014011759, -0.02741646, 0.026385874, 0.054409392, -0.024112932, 0.021388227, 0.026315285, -0.008555291, -0.015402347, -0.011484702, 0.033571754, -0.00010637864, -0.017138816, 0.031143518, 0.0035911752, 0.012007055, -0.042663515, 0.0013305878, 0.01023529, 0.0067658797, 0.003395293, -0.015359994, -0.04387763, -0.008167055, -0.0053294096, 0.020188227, 0.006716468, 0.0144211715, 0.040602338, -0.021204699, 0.007461174, 0.0045176456, -0.014004701, 0.010298819, 0.016969405, 0.021811757, -0.010771761, -0.03252705, 0.010743526, -0.00012606612, -0.05183998, 0.044865865, 0.009974114, 0.008195291, -0.012112937, 0.012585877, 0.008957644, 0.020583522, 0.005604704, -0.0049129394, -0.012762348, -0.0067552915, 0.0052235276, -0.0069352915, -0.04291763, 0.0029435284, 0.020795286, 0.021642344, -0.024691755, -0.029110577, -0.0007972056, -0.018818816, 0.075444676, 0.005191763, 0.006790586, -0.01802823, 0.0024105874, -0.010559996, 0.001266176, -0.016715288, -0.030381165, 0.01939764, -0.036564693, -0.019623522, 0.009331761, 0.030550577, 0.0031923517, 0.011957643, 0.0028994107, 0.0045070574, -0.08719055, 0.0065823505, 0.010397643, 0.00820235, -0.003765881, 0.040574104, -0.021204699, -0.033007048, -0.018635288, 0.013404701, -0.015345877, 0.017802346, -0.038287044, -0.020964699, -0.013609407, -0.020202346, 0.011682348, -0.02987293, -0.0011338231, 0.005407057, -0.0122258775, -0.04221175, -0.008555291, -0.015331759, -0.014527054, 0.01657411, -0.0141952885, 0.03998116, -0.0058941157, -0.081035264, -0.008604703, -0.0061588213, 0.011767055, -0.0098823495, -0.01359529, 0.005075292, -0.0025941166, 0.059407037, 0.013207054, -0.027755285, -0.015543523, -0.008901173, -0.0033917634, 0.0069847032, -0.03557646, 0.025454108, -0.0074752914, -0.008209409, -0.0180847, -0.06070586, -0.02209411, 0.014047054, -0.0023664697, -0.002687646, 0.011576466, -0.00999529, 0.022291757, -0.023392932, -0.029195283, 0.007969409, 0.018536463, 0.021769404, 0.02551058, -0.03441881, -0.0013579407, 0.007919997, -0.015967052, 0.0045635276, 0.007976468, -0.019115288, 0.015995288, -0.023971755, 0.01850823, 0.013602348, 0.010905879, -0.0031976458, -0.0014770583, 0.016461171, -0.031341165, -0.017887052, -0.021698816, 0.013397642, 0.001909411, -0.008251762, 0.0380894, 0.013122348, -0.009070585, -0.008823526, -0.025058813, 0.015430583, 0.016503524, -0.0074329386, 0.011082348, 0.026484696, -0.03035293, -0.020371757, 0.006141174, 0.01017882, 0.009303526, 0.054155275, 0.00036992633, 0.019567052, 0.038964693, 0.003988234, 0.030861165, -0.023223521, 0.037044693, -0.009049408, -0.013482348, 0.00087220554, 0.0028588225, -0.00077514676, -0.054861158, 0.015402347, -0.016362347, -0.0011602937, -0.022757638, -0.0070694094, -0.02023058, -0.032894105, 0.009889408, 0.01771764, 0.019835288, -0.021119991, -0.001814117, 0.010637643, 0.0028958812, -0.10605173, 0.0037411752, 0.031228224, -0.022164697, -0.023816463, -0.011167055, 0.024734108, 0.008999997, -0.012338819, -9.816173e-05, 0.010898819, -0.009077644, 0.011534113, -0.01969411, 0.012967054, -0.021599991, 0.005622351, -0.025171755, 0.034983516, -0.03526587, -0.0071752914, 0.0056788214, -0.023237638, -0.013404701, -0.044724688, 0.009832937, -0.030804694, -0.0056541157, 0.010694114, 0.031199988, -0.019256463, 0.012501172, -0.025101168, 0.0073482324, -0.016461171, 0.0145411715, -0.008971761, 0.0007861762, 0.008901173, 0.0077717616, 0.039529398, -0.0051599983, -0.004944704, -0.027684696, 0.009790584, -0.034701165, -0.0126776425, 0.0030758812, 0.017802346, 0.019623522, 0.0016332347, 1.8791354e-05, 0.011322348, 0.017039994, 0.020301169, -0.029195283, 0.014625877, -0.009501173, 0.003167646, 0.04111057, 0.04514822, 0.05167057, 0.01358823, -0.01299529, -0.0050188215, -0.0053188214, -0.0075882324, 0.018776463, -0.0453741, -0.04370822, -0.037242338, 0.016969405, -0.0105882315, 0.020978816, 0.012162348, 0.015190583, 0.025849402, 0.017858816, -0.0015776465, -0.004108234, 0.055708215, -0.018451758, -0.006878821, 0.029759988, -0.0075952914, 0.038343515, 0.010842349, 0.017167052, 0.003670587, 0.05178351, 0.050795276, -0.038767044, -0.009797644, 0.027148224, -0.057487037, -0.026315285, 0.024522344, 0.014851759, 0.030663518, 0.0009952937, 0.006896468, 0.005844704, -0.01826823, -0.023181168, -0.030578813, 0.0077788206, 0.003077646, -0.007828233, -0.023872932, -0.000723088, 0.027741166, 0.035237633, -0.009395291, 0.0366494, -0.0042423513, 0.03681881, 0.03532234, -0.00940235, -0.0039599985, -0.021416463, -0.0002644852, 0.029703518, -0.005131763, -0.0021829405, 0.0030811753, 0.04291763, -0.01753411, -0.0017073523, 0.008089409, 0.0077082324, 0.027684696, -0.0009926467, -0.045063514, -0.019962346, -0.017816463, 0.003275293, -0.009663526, -0.0012670583, -0.005407057, 0.014823524, -0.04071528, -0.003705881, 0.026174108, 0.0007147056, -0.061891742, -0.048564687, -0.01634823, -0.007447056, -0.01418823, -0.009861173, 0.011414113, 0.042635277, 0.004598822, 0.020583522, -0.0012794113, -0.005611763, 0.05006116, 0.0058799977, -0.009967055, 0.058447037, 0.013736466, 0.026894107, 0.015232936, 0.009352937, -0.004468234, 0.0096564675, -0.013009407, -0.0013491171, -0.0052905865, 0.0059399977, -0.040207043, -0.0030282342, -0.0016808817, 0.030861165, 0.02939293, 0.0074823503, -0.00020536757, -0.0019023522, -0.0138494065, 0.02735999, 0.008004703, 0.021825874, -0.006557645, -0.028390577, -0.009317644, -0.010256466, 0.021232933, 0.016390583, 0.023661168, -0.0166447, -0.003280587, 0.02311058, -0.010009408, 0.019185875, -0.03605646, -0.0028270578, 0.014287054, 0.021105874, -0.025312932, -0.03458822, -0.061496448, 0.049327042, 0.022588227, -0.018748228, 0.035011753, 0.023858815, -0.016856464, 0.017011758, -0.0055164685, 0.024762344, -0.0173647, 0.007076468, -0.010644702, 0.0111741135, -0.033797633, -0.0045670574, -0.017618816, 0.008265879, 0.01825411, -0.020625874, -0.059971742, -0.026752932, -0.022108227, 0.01969411, 0.0033776457, 0.021585874, 0.030861165, -0.016277641, 0.0042423513, 0.0035876457, -0.041816454, -0.029901166, -0.023929402, 0.009381173, -0.024056463, 0.03334587, -0.042014103, -0.011477643, -0.004951763, 0.020498816, 0.054974098, 0.005347057, -0.018183522, -0.01814117, -0.00049764686, -0.015585877, 0.025397638, 0.019482346, 0.0034147047, -0.02161411, -0.025595285, -0.002669999, 0.027755285, -0.014527054, 0.019383522, -0.00061323505, 0.04627763, 0.023491757, 0.04895998, 0.0073552914, -0.022828227, 0.032470576, 0.0065152915, -0.03735528, 0.0034976457, -0.0011144114, -0.022884697, -0.011322348, 0.03687528, -0.014879994, 0.025905874, 0.038399987, 0.006741174, 0.03201881, -0.010411761, -0.009705879, 0.014767054, -0.00808235, 0.009303526, -0.016150583, 0.020682346, 0.025962343, -0.018988227, 0.0026152933, -0.0061199977, -0.024522344, 0.0055164685, 0.0041294103, -0.0045176456, 0.001831764, -0.017999994, 0.0029311753, 0.0070976443, 0.0037129398, -0.003513528, 0.002082352, 0.010679996, 0.010143526, 0.013122348, -0.03777881, 0.010199997, -0.0031094106, 0.025891755, -0.03334587, -0.014138819, 0.011237643, 0.026809402, -0.045487043, -0.003945881, 0.014209406, -0.016122347, 0.02891293, 0.005495292, 0.019002346, -0.016475288, -0.030127048, -0.012437643, -0.02987293, 0.010185879, -0.013814112, -0.02573646, 0.018239994, -0.003921175, 0.03537881, -0.018338816, -0.03218822, -0.0029594107, -0.0039670574, -0.008534115, 0.0070094094, -0.00018794111, -0.0065823505, -0.013736466, 0.014075289, -0.015331759, 0.020527052, -0.014061172, -0.0042952923, 0.006794115, -0.008689408, -0.022799991, 0.020597639, -0.019425875, 0.0004914704, 0.0051494096, -0.0041294103, -0.011251761, -0.016249405, 0.013679995, 0.01771764, 0.08990114, -0.053308215, -0.020159993, -0.024889402, 0.004867057, 0.0032717634, -0.015487053, -0.02460705, -0.06358586, -0.01269176, -0.028362343, 0.016602347, 0.0071223504, 0.0042599984, 0.0031394106, 0.015995288, -0.007376468, 0.0057988213, 0.033571754, 0.008068232, 0.01730823, 0.050428215, -0.027698813, 0.0032082342, 0.006938821, 0.012239995, 0.019072935, 0.027303519, 0.007439997, 0.026018813, -0.008872938, -0.03794822, -0.010143526, -0.0016049994, -0.045119982, 0.00088191143, 0.020131757, 0.029364696, -0.018381169, -0.0039176457, -0.027571755, 0.015049406, 0.00873882, 0.0006383821, 0.007616468, 0.010157643, 0.0062647033, -0.0059999977, 0.005435292, -0.021077638, -0.028644696, -0.017816463, -0.024381168, -0.013743524, 0.046898805, -0.036959987, -0.0069423504, 0.023025874, 0.0013844112, -0.0070199976, 0.0009705879, 0.043849397, -0.01843764, 0.0054388214, -0.012974113, -0.05477645, -0.02268705, 0.010214114, -0.0048705866, 0.0150070535, -0.051698804, -0.013432936, 0.021952933, -0.014392936, 0.013369407, -0.00044492632, -0.014336465, -0.02239058, 0.040432926, 0.011110584, 0.0005713233, -0.040404692, -0.034672927, -0.018395288, -0.009564702, 0.00042242632, 0.04159057, -0.0065823505, -0.03998116, -0.0028694107, -0.019976463, 0.013157642, 0.041392926, 0.047491748, 0.012790583, -0.025934108, -0.017491758, -0.017943522, 0.010828231, -0.01987764, 0.013072937, -0.009557644, -0.017096464, 0.001403823, -0.024451755, -0.023901168, -0.051388215, 0.015515288, 0.020512933, -0.00043477924, 0.006737645, -0.0026241166, -0.004944704, 0.015119994, -0.006861174, 0.020498816, -0.004196469, 0.06674821, -0.031171754, -0.03602822, 0.0067658797, -0.0060529388, -0.004228234, -0.0016888229, 0.007757644, -0.0111741135, 0.023816463, 0.007531762, -0.0006397056, 0.0058694095, 0.015331759, 0.0019464699, 0.02401411, -0.0010120585, -0.007651762, -0.010496466, -0.012416466, -0.0046235276, -0.035632927, 0.02939293, -0.008145879, 0.0144211715, -0.027995285, -0.043087043, 7.196688e-05, 0.025552932, 0.014597641, -0.010602349, 0.013524701, -0.016418818, -0.026004696, -0.008294115, 0.004524704, 0.001831764, -0.001874117, -0.03580234, -0.0152470535, -0.001041176, 0.015727052, 0.011364702, 0.03458822, 0.0032311752, -0.024381168, -0.0004021322, 0.011830584, -0.008654115, -0.033797633, 0.0020082346, 0.048112925, 0.01107529, -0.01897411, -0.01274823, -0.016221171, 0.006734115, -0.0029188225, -0.01239529, 0.0024758815, 0.030381165, 0.0043729395, 0.037750576, -0.017378816, -0.005371763, -0.0017514699, -0.019129405, -0.024917638, 0.016842347, -0.021712933, -0.002562352, -0.010192937, 0.018310582, 0.004902351, -0.032159988, -0.005537645, 0.003395293, 0.034334105, 0.034870576, -0.015783524, 0.028630577, 0.024691755, -0.019595288, 0.019425875, -0.0061588213, -0.015374112, -0.031454105, -0.027021166, 0.036084693, -0.005470586, -0.03585881, 0.0057317624, 0.018310582, -0.0040305867, 0.0044258805, -0.0020276464, -0.018959993, 0.021938816, -0.021656463, 0.0031694106, -0.007531762, -0.0075105852, -0.023449402, -0.025905874, 0.01826823, 0.03933175, 0.0037094103, -0.025369402, -0.002592352, -0.022291757, -0.02263058, -0.0028870578, -0.008985879, 0.0045035277, -0.02669646, -0.026301166, 0.023661168, -0.0008077938, 0.019030582, -0.03837175, -0.008830585, 0.010079997, 0.022376463, 0.017491758, 0.02065411, -0.0006383821, 0.018861169, -0.0044999984, 0.037327044, 0.005339998, -0.028263519, -0.004365881, -0.02401411, 0.028941166, 0.008901173, -0.017195288, -0.015430583, -0.014159995, -0.026851755, 0.01406823, 0.0058764685, -0.02532705, 0.0041894102, -0.057628214, -0.020795286, 0.0027564697, 0.011230584, 0.0007208821, 0.014738818, 0.0018388228, -0.010185879, -0.040969398, 0.03794822, -0.0056329393, -0.011131761, -0.012642348, -0.018451758, 0.030804694, 0.03789175, 0.011442348, -0.003393528, -0.00975529, -0.029138813, 0.009515291, -0.022517638, -0.0050364686, -0.014209406, -0.010037643, 0.014258819, 0.0075105852, 0.014470583, 0.020992933, 0.00010935658, -0.016941171, 0.031397637, 0.006501174, 0.0185647, -0.0138494065, 0.0074188206, -0.0013826465, -0.016517641, -0.009232937, 0.0030899988, -0.033374105, -0.0068752915, 0.0035752929, -0.018211758, 0.009917644, 0.030691754, -0.014174112, 0.03526587, -0.015063524, -0.016912935, -0.003555881, -0.024903521, -0.008872938, 0.0032735283, 0.021698816, 0.013602348, 0.017209405, -0.019468227, -0.027585872, 0.0054388214, 0.0025888225, 0.029675283, -0.021162346, -0.011788231, 0.0026947048, -0.0069176443, -0.026922343, 0.036423516, 0.0052941157, 2.7380505e-05, -0.0028870578, 0.023576463, 0.023985874, 0.015303524, -0.0038858808, -0.008103526, -0.019863522, -0.02071058, 0.016418818, -0.012254113, -0.017449405, -0.0073623504, 0.010552937, 0.006787056, 0.017816463, 0.028390577, -0.015303524, 0.016517641, -0.024945874, 0.005541174, -0.0035999988, 0.017322347, 0.009543526, -0.009070585, -0.0023205874, -0.0025835284, -0.0034764693, 0.008364703, 0.012719995, -0.057035275, -0.042776454, 0.0023929402, 0.016983524, 0.022997638, -0.011329408, 0.020159993, -0.020724699, -0.050625864, -0.026414108, 0.002472352, -0.022418816, 0.007447056, 0.028404696, -0.012614113, -0.029929401, -0.006077645, 0.0010279408, -0.017209405, 0.010651761, -0.027967049, 0.021811757, -0.026329402, 0.027571755, -0.012762348, 0.024649402, -0.0144352885, 0.00014944847, 0.017167052, -0.010644702], "content_hash": "9a82f9c41e8422d7a6ef098b8129b12e"}, {"content": "# Experiment: Your First Analysis\n\n## Goal\n\nLearn how to run your first analysis and walk through each pipeline stage to understand how a transformer model processes text.\n\n## Prerequisites\n\nNone -- this is the starting experiment.\n\n## Steps\n\n### Step 1: Select a Model\n\n1. In the **generator section** at the top, find the \"Select Model\" dropdown.\n2. Choose **\"GPT-2 (124M)\"** from the list.\n3. Wait for the model to load. You'll see a status message indicating the model is ready.\n\n### Step 2: Enter a Prompt\n\n1. In the **\"Enter Prompt\"** textarea, type: `The cat sat on the`\n2. Leave the generation settings at their defaults (1 beam, a few tokens).\n\n### Step 3: Run the Analysis\n\n1. Click the **\"Analyze\"** button.\n2. Wait for the analysis to complete. The pipeline stages and generation results will appear.\n\n### Step 4: Explore the Generated Sequences\n\nLook at the **generated sequence(s)** below the generator. You should see how GPT-2 continues your prompt. Common completions might include \"mat,\" \"floor,\" \"bed,\" or similar words.\n\n### Step 5: Walk Through the Pipeline\n\nNow expand each of the **5 pipeline stages** by clicking on them:\n\n**Stage 1 - Tokenization**: Click to expand. You'll see your prompt split into tokens. Notice how each word (and its leading space) becomes a separate token. Count the tokens -- \"The cat sat on the\" should produce about 5 tokens.\n\n**Stage 2 - Embedding**: Click to expand. You'll see that each token was converted into a 768-dimensional vector. This is GPT-2's hidden dimension.\n\n**Stage 3 - Attention**: Click to expand. This is the richest stage:\n- Look at the **head categories**. You should see heads grouped into Previous-Token, First/Positional, Bag-of-Words, Syntactic, and Other.\n- Click on a category (like \"Previous-Token\") to see which specific heads belong to it.\n- Below the categories, you'll see the **BertViz visualization**. Try clicking on individual head squares to see their attention patterns.", "source_file": "experiment_first_analysis.md", "chunk_index": 0, "embedding": [-0.0011164723, 0.029201265, 0.017076392, -0.04438592, -0.02089795, -0.04204982, -0.005459367, 0.03229914, 0.026890555, 0.07973215, 0.05101334, -0.023919646, -0.022091392, -0.038545668, 0.007909734, 0.020453582, -0.0002761424, 0.022776987, 0.0068559493, 0.07912273, 0.037149087, -0.026712809, -0.0059799114, 0.036488887, 0.029099695, -0.029861467, -0.00778912, 0.048753418, 0.00097046595, -0.026585847, 0.018739594, -0.010658462, -0.058961164, -0.011794772, 0.011128222, -0.005326057, -0.03897734, 0.030750202, -0.007643114, 0.010201398, 0.0071479618, -0.0142451385, -0.020986822, 0.0095031075, -0.006014826, 0.034406707, 0.011305968, -0.03095334, -0.020783683, 0.011705899, -0.01430862, 0.03940901, 0.0038342532, -0.0059830854, -0.008963519, -0.024948038, -0.038494885, -0.022776987, -0.03547319, -0.03359415, 0.011915386, 0.0110329995, 0.023361012, 0.04456367, 0.006970215, -0.010867949, -0.037961643, 0.022802379, 0.0030470886, 0.01955215, 0.026636632, 0.004640462, -0.028160177, 0.0055355444, 0.0073130126, -0.012772379, -0.028718809, 0.036488887, -0.025659025, -0.034889165, 0.012251835, -0.020237746, 0.00038941632, -0.030394709, 0.0076177213, -0.0007947029, -0.028693417, -0.02249767, -0.041262656, -0.04235453, 0.004100873, 0.016898645, -0.05271463, 0.029404404, 0.036260355, 0.0070400443, -0.0102394875, -0.0114011895, -0.008379494, 0.03283238, 0.050810203, -0.0054466706, 0.0005832318, 0.012600981, 0.020047303, -0.019767987, 0.023018215, 0.014219747, -0.056421924, -0.034812987, -0.063938074, -0.02419896, 0.013356404, 0.0049388227, 0.038063213, 0.02620496, -0.006722639, -0.0021043955, 0.0053673196, -0.047585364, -0.026154177, -0.015959127, -0.008703247, -0.010309316, -0.032375317, -0.021812076, 0.023716506, -0.00640206, 0.022738898, -0.009566588, -0.020936038, 0.026865164, 0.01835871, -0.025697114, -0.021405797, -0.013242139, -0.023462582, 0.026915949, -0.025468582, 0.029353619, 0.008265228, -0.020415492, -0.01885386, 0.038596455, -0.05418739, -0.061398834, -0.01614957, 0.064750634, -0.043573365, 0.016073393, 0.033035517, 0.024579847, -0.03791086, -0.025189266, 0.01952676, -0.03981529, 0.004916604, 0.01818096, 0.002445606, -0.060078427, 0.03519387, 0.016200354, -0.03773311, -0.008430278, -0.046671238, 0.04596025, -0.032984734, 0.003453367, -0.027957037, -0.057183694, 0.00037295095, -0.02159624, 0.002205965, -0.030369315, 0.02079638, -0.00066893117, -0.10385493, -0.07038774, -0.025367012, -0.0701846, -0.01680977, -0.00065821875, -0.032680023, -0.019183962, -0.03656506, -0.026255745, 0.018295227, 0.0068749934, 0.0047928165, 0.016124176, 0.0045547625, 0.02650967, 0.039662935, 0.0022313576, -0.006728987, -0.0073384047, 0.07262228, 0.01668281, 0.025163872, 0.030089999, 0.04773772, 0.023157872, 0.008976215, -0.015933733, 0.023310227, 0.027119087, -0.03367033, 0.013902341, -0.061652757, -0.036895163, 0.0377839, -0.061449617, 0.031181872, 0.016898645, -0.056167997, -0.028134784, -0.04700134, 0.015324316, 0.043852683, 0.05926587, 0.024884555, 0.025125785, 0.02430053, 0.012042347, 0.0041103954, 0.034559064, 0.026890555, -0.053425618, 0.02477029, -0.010201398, 0.015552848, 0.016835164, -0.015057696, -0.008163658, 0.014346709, -0.034355924, 0.031511974, 0.022015214, -0.022637328, -0.009903038, -0.0052594016, 0.009382494, 0.027119087, 0.009249183, 0.054035038, 0.018866556, -0.03389886, -0.07495838, 0.051698934, 0.06368415, 0.016162265, 0.002707465, -0.026433492, 0.03346719, -0.020148873, -0.004459541, -0.020440886, 0.017165266, 0.0022265965, -0.01143293, 0.008798468, 0.0041611805, -0.035981037, -0.019006215, -0.0076494617, -0.023361012, -0.037199873, -0.03232453, -0.010772727, -0.053171694, -0.0055514146, 0.004205617, 0.020440886, 0.005084829, -0.061449617, -0.008639766, 0.005097525, -0.011109177, -0.0069257785, 0.031613544, -0.044614453, -0.0021567673, 0.04055167, 0.05322248, -0.018561848, 0.019729897, 0.022218354, -0.009541196, 0.0053736675, -0.021964429, 0.0591643, -0.015692506, 0.03189286, -0.02650967, -0.004434149, 0.028083999, -0.004205617, 0.01889195, -0.019336317, 0.039764505, 0.034914557, -0.036031824, 0.054898378, 0.014702202, 0.018904645, 0.02861724, -0.017266834, -0.030089999, 0.012931082, 0.009814165, 0.0063068382, -0.018269835, 0.0041167433, -0.029277442, 0.017216051, 0.018396797, 0.01741919, 0.0026662024, -0.006532196, -0.010410886, -0.03283238, -0.041872077, 0.037428405, 0.0054879333, -0.0059481706, 0.03463524, -0.014994215, 0.019310923, 0.025468582, 0.012366101, 0.023107087, -0.024224354, -0.0056117214, 0.03600643, -0.04192286, -0.035701722, -0.016632024, 0.06617261, 0.028033214, 0.041262656, -0.09816703, -0.005694247, -0.0102394875, -0.01230262, -0.06581711, 0.07338405, 0.01293743, 0.00889369, 0.012810468, -0.013953126, -0.03824096, 0.00047452055, -0.0047356836, -0.018980822, 0.036260355, 0.034127392, -0.011737639, -0.0020298052, 0.00075224997, -0.022002518, -0.019133177, 0.02517657, 0.05428896, -0.018447582, -0.005141962, 0.0223961, -0.006398886, 0.023081696, -0.008652462, -0.011071089, -0.0021535933, 0.026154177, 0.020567847, -0.023018215, -0.009319012, -0.01617496, -0.017089088, -0.013623025, -0.027398404, 0.07749762, -0.021913646, 0.033848073, -0.035955645, -0.02968372, 0.025201961, 0.042126, 0.0024059303, 0.008411234, -0.04339562, 0.0404501, -0.011045696, -0.047483794, -0.058859594, 0.015425886, -0.016720898, 0.03483838, -0.010328361, 0.022421492, 0.059621364, 0.03813939, 0.007985911, -0.026915949, -0.03633653, -0.019298228, -0.008328709, 0.024110088, 0.036895163, 0.03133423, 0.024744898, 0.015324316, 0.04763615, -0.008138265, 0.017914342, -0.04362415, -0.014448278, 0.024821075, 0.03372111, 0.034508277, 0.011020304, 0.024224354, -0.029379012, -0.06830557, -0.024998821, -0.0018742769, -0.06637575, 0.015794076, 0.03697134, -0.013153265, -0.0038723417, -0.025443189, -0.010379145, 0.009147613, 0.03798704, -0.0115345, -0.016657418, -0.031131087, 0.026179569, 0.06881341, 0.02159624, -0.020453582, 0.016936734, -0.019412493, -0.033441797, -0.017152568, -0.043674935, 0.021266138, -0.011394842, -0.06795008, -0.013673809, -0.037022125, -0.0019393449, 0.0027772943, 0.002702704, 0.0033549715, -0.0012005846, -0.019882252, 0.027423797, -0.0015346535, 0.10009686, 0.007858949, 0.0042754463, 0.013089784, -0.0010521978, 0.00013688093, -0.0058719935, 0.010709247, -0.041567367, -0.028820379, 0.020440886, -0.02075829, -0.006659158, -0.023157872, -0.00879212, -0.02560824, -0.007160658, -0.020313924, 0.012772379, 0.02075829, -0.013038999, -0.02788086, 0.009814165, 0.0075415443, 0.038545668, 0.044233568, -0.023653025, 0.07302856, -0.060281567, 0.0005586329, 0.014956126, -0.017368404, -0.0213931, -0.013013607, 0.028896555, -0.0135595435, 0.018777683, -0.0021123306, -0.0059386487, -0.023538759, -0.008062088, 0.018269835, -0.006456019, -0.013699202, 0.0076748542, 0.059621364, 0.0054308004, -0.017863557, 0.029404404, 0.045325443, -0.0076367655, 0.040907163, -0.05007382, 0.010658462, -0.017152568, 0.010309316, 0.0013616676, 0.037225265, 0.020720202, 0.027703114, 0.013978519, 0.0016711376, -0.0007220965, 0.0037898163, 0.011712247, -0.008976215, -0.019729897, -0.026712809, -0.03821557, -0.0785641, -0.031283442, -0.011159962, 0.03105491, 0.0010307729, 0.00051379943, 0.013546848, -0.048067823, -0.0067416835, 0.016200354, -0.024922645, -0.06662967, 0.002755076, -0.01774929, 0.016022608, 0.023335619, -0.002188508, -0.0015283054, 0.010880645, 0.028490277, 0.0014449866, 0.015387797, 0.019945733, 0.036285747, -0.03940901, 0.02497343, -0.012702551, -0.01955215, 0.0056720283, 0.005015, -0.03801243, 0.011617025, -0.041288048, 0.00879212, 0.0048689935, 0.02292934, -0.021773987, -0.009325361, 0.05555858, 0.025659025, -0.026636632, -0.011839208, -0.0083985375, 0.01965372, -0.006532196, 0.024744898, -0.007459019, 0.012892993, -0.032172177, -0.030267745, 0.018739594, 0.0033073607, 0.0042373575, -0.016619328, -0.0058910376, -0.02059324, 0.027169872, -0.009027, -0.0042024427, 0.010614025, 0.0041707023, -0.01881577, -0.026433492, 0.047940858, -0.008341405, -0.007839905, 0.010779075, -0.04309091, -0.01394043, 0.013242139, -0.016657418, -0.03747919, 0.012429582, 0.031867467, -0.009991911, -0.03623496, 0.022586543, -0.02958215, 0.010404537, -0.02524005, 0.002894734, 0.010690202, -0.0037136392, -0.035727113, -0.011477367, -0.03222296, -0.005456193, -0.026331924, 0.025392404, -0.016048, -0.007916082, 0.0027979256, -0.007776424, 0.012213746, -0.02079638, 0.011394842, -0.03613339, 0.012264531, 0.00022317543, -0.006728987, -0.009331709, 0.010493411, 0.03234992, -0.00035231962, 0.04326866, 0.03928205, -0.006716291, -0.013724594, 0.004008826, 0.019514063, -0.015425886, -0.0031930949, -0.027626935, -0.0020329794, -0.023145176, -0.007198747, -0.017596936, 0.004472237, 0.018561848, 0.002829666, 0.027931646, -0.028058607, -0.024427492, -0.064750634, -0.02089795, 0.06525848, -0.0008058121, 0.02704291, -0.009096829, -0.0040469146, -0.047712326, 0.0035549367, -0.0064877593, -0.042278353, 0.026458886, -0.023462582, -0.011648766, -0.010360101, 0.019844163, 0.010906038, 0.022180265, 0.020313924, -0.0049769115, -0.07186051, -0.012175658, 0.011909038, -0.009852253, 0.0067797718, -0.022154873, -0.00032117424, -0.058250174, 0.014676809, 0.026738202, 0.0036215917, -0.0004042947, -0.029734505, 0.022345316, 0.008284272, -0.006602025, 0.0041675284, -0.021608936, 0.015882948, -0.005456193, -0.009223791, -0.018206354, -0.010480715, -0.012378797, -0.0044944556, 0.0052371835, -0.028591847, 0.04890577, -0.02102491, -0.08978754, -0.04014539, 0.0007736748, -0.018003214, 0.0037453796, -0.007160658, -0.019361708, -0.03115648, 0.04593486, 0.028998125, -0.03062324, -0.02694134, -0.010042696, 0.010182355, 0.022548456, -0.024986126, 0.0097697275, -0.0149307335, -0.011693202, 0.0007367765, -0.025836771, -0.020415492, 0.008404885, -0.022408796, 0.024160873, 0.0036660284, -0.052054428, 0.0013965822, -3.394746e-05, -0.022218354, 0.0424561, 0.03697134, -0.004716639, 0.014879949, -0.04730605, 0.023373708, 0.00876038, 0.006881342, 0.01948867, -0.007141614, -0.021939037, 0.00072328676, 0.012550196, 0.0077065947, 0.038317136, -0.0041199178, -0.030699417, -0.022561152, 0.042836986, 0.003958041, -0.033517975, -0.008665158, 0.010207746, -0.008804816, -0.03105491, 0.017063696, 0.007528848, 0.025519367, -0.019945733, -0.016048, 0.037352227, 0.00064512575, 0.0027407927, -0.012061392, 0.024389405, -0.010639417, -0.0061862245, 0.04235453, 0.022319924, 0.011877297, 0.0035422405, 0.020707507, -0.0065702847, 0.03750458, -0.01835871, 0.03392425, -0.018333316, 0.051927466, 0.006862297, -0.014676809, -0.0015417951, -0.009236488, 0.026814379, -0.02777929, -0.033543367, 0.025646329, -0.0032327706, -0.017711202, -0.007268576, -0.03666663, -0.011693202, -0.024554456, -0.02714448, 0.032959342, -0.014321316, -0.014067392, -0.0056244177, -0.02450367, -0.115281515, -0.0012632721, -0.002393234, -0.02915048, 0.0013862666, -0.034432102, -0.0009664984, 0.023081696, -0.0464681, -0.017127177, 0.015819468, -0.019463278, -0.0017314446, 0.0058751674, 0.022484975, -0.046874378, 0.020986822, -0.04034853, 0.012454974, -0.022942038, -0.007827208, -0.0059323004, 0.0009791947, -9.308895e-05, -0.04573172, -0.011648766, -0.007382842, 0.012620025, 0.025900252, 0.015755987, -0.03250228, 0.03631114, -0.014105481, 0.010309316, 0.012042347, 0.017508063, -0.034051213, 0.017876253, 0.04329405, -0.016974822, -0.0018885601, -0.009680854, 0.025557455, -0.009541196, 0.047890075, 0.016657418, -0.0123470565, -0.003634288, 0.008220791, 0.023107087, 0.012543848, 0.014499063, 0.021266138, -0.011077437, 0.033340227, 0.023462582, -0.009642766, -0.029505974, 0.022015214, 0.04298934, -0.028896555, 0.032933947, 0.042557668, 0.008487411, -0.0068115126, 0.020542456, -0.019666417, 0.025722506, -0.026357315, -0.04222757, -0.02237071, -0.017508063, -0.00064869656, -0.015629025, -0.018879253, -6.144764e-05, 0.0002561062, 0.035828684, -0.03336562, -0.01611148, 0.02477029, -0.0026836598, -0.016263835, 0.015832163, 0.008804816, 0.03199443, 0.00070344895, -0.025367012, 0.0021202657, 0.0417959, 0.015057696, -0.02212948, -0.009001607, 0.01578138, -0.02286586, -0.023640329, 0.01243593, 0.04372572, 0.015933733, -0.0028122088, 0.007592329, 0.02036471, -0.05545701, -0.008976215, -0.03651428, 0.02450367, 0.0075796326, 0.0033994082, -0.03199443, -0.015667114, 0.023627631, -0.0008839731, 0.0203901, 0.014156265, -0.015768683, 0.047179088, 0.06596947, -0.0058910376, -0.010487063, -0.03694595, 0.0121883545, -0.008163658, -0.01684786, 0.00939519, -0.0544921, 0.0024424319, -0.046391923, -0.0060021295, -0.0350923, -0.015552848, 0.009680854, 0.018942734, -0.016428886, -0.034482885, 0.012778727, 0.0051260917, -0.009630069, 0.0066210693, -0.018003214, -0.003951693, -0.010633069, 0.026484277, 0.018968126, 0.0046848985, -0.023475278, -0.007928778, -0.010886993, -0.03326405, -0.0077827717, 0.0223961, 0.02567172, 0.0053768414, 0.007211443, 0.008639766, -0.003945345, -0.012023304, -0.010448975, -0.026052607, -0.01143293, 0.014486367, 0.0048499494, 0.028896555, 0.030496277, -0.0060687847, 0.008271576, -0.03534623, 0.00031304074, -0.005088003, -0.03252767, 0.013038999, -0.01922205, 0.06998146, 0.028896555, 0.015806772, 0.022942038, 0.029328227, 0.009230139, 0.02159624, 0.011807468, 0.04489377, 0.0051292656, 0.017888948, -0.0051070475, -0.0031089825, 8.26369e-06, -0.019349013, -0.015451279, 0.013648418, -0.03519387, 0.012156613, -0.01818096, 0.025900252, -0.01832062, 0.037225265, -0.004894386, -0.0014632373, 0.027804682, -0.014778379, -0.020478975, -0.02317057, -0.021837467, 0.007592329, 0.016035303, -0.041034125, 0.01013157, 0.018980822, -0.012531152, 0.018295227, 0.0004741238, -0.013991215, 0.0016441582, -0.007909734, -0.02022505, -0.007909734, -0.018066695, -0.0060433922, 0.0036215917, 0.032806985, 0.0051070475, -0.014511759, -0.07069246, -0.049997643, -0.021380404, -0.0049388227, -0.021202657, 0.0024773465, 0.035117697, -0.01774929, -0.007084481, -0.004002478, -0.024465581, -0.039332833, 0.00219803, -0.0012275641, -0.0057005947, 0.027423797, 0.00491343, -0.00019530329, 0.016086088, -0.009534848, 0.03613339, 0.009623721, 0.023475278, -0.009477715, 0.010099829, -0.01691134, 0.05312091, -0.0034057563, 0.009947474, 0.003513674, -0.0085572405, 0.051902074, 0.013965823, -0.0123470565, 0.026052607, 0.0076304176, 0.018460277, 0.025189266, -0.005652984, -0.0026598542, -0.0043008383, 0.033949643, 0.0065004556, -0.053273264, -0.011210746, -0.032578453, -0.020834468, -0.009376145, 0.027423797, 0.015235443, 0.009579284, 0.029759899, -0.009217443, 0.03105491, -0.00849376, -0.035016127, 0.007935126, -0.028210962, 0.02423705, 0.006195747, -0.005195921, 0.014460974, 0.010734639, -0.01832062, -0.0072050947, -0.018168265, -0.045274656, -0.016238442, -0.014702202, -0.0073955376, 0.008379494, -0.012962823, -0.02788086, -0.018206354, -0.0005312567, 0.003885038, 0.026966734, 0.007097177, -0.00024856784, -0.024186265, -0.014397494, 0.00012487905, 0.018104784, -0.059316657, -0.005779946, 0.021304227, 0.0086080255, -0.07566936, -0.023297532, 0.0010783836, -0.008684202, 0.008931778, 0.012702551, 0.02704291, 0.0049991296, -0.008265228, 0.005564111, 0.0015275119, 0.0017219224, -0.02012348, -0.03930744, 0.013064392, 0.008195398, 0.03367033, -0.0069511705, -0.021545455, 0.018599937, -0.0033771899, 0.02724605, -0.0017092262, 0.034508277, -0.0113504045, 0.010537848, 0.011477367, -0.014626025, -0.0038056867, -0.035549365, -0.018231746, 0.0117185945, -0.009077785, 0.022383405, 0.007274924, -0.026636632, 0.01089969, 0.016390797, -0.011274228, 0.015514759, -0.01013157, 0.02186286, 0.030597847, 0.058808807, -0.022789683, -0.027169872, 0.004075481, 0.0038374271, -0.010042696, -0.0044881073, -0.013648418, -0.028439494, -0.00049316813, -0.015755987, -0.00034359097, -0.017596936, 0.011851905, 0.0067035947, 0.013432582, -0.006906734, -0.016009912, 0.05830096, 0.011496411, 0.02386886, 0.059113517, -0.016530456, -0.024097392, 0.015451279, 0.026865164, -0.00034557475, -0.01069655, 0.0065766326, 0.026433492, 0.018561848, -0.04596025, -0.0053070127, 0.01009348, -0.028769594, 0.029531367, 0.0058529493, 0.016225746, -0.037123695, 0.0014981518, -0.0063544493, 0.018561848, 0.026788987, -0.022637328, 0.017254138, 0.030064607, 0.01494343, -0.008976215, -0.011528152, -0.030547062, -0.021189962, 0.0007518532, 0.0019060173, -0.031004125, 0.041211873, -0.043852683, 0.009757032, 0.0016774858, 0.024046607, -0.00056974206, -0.037199873, 0.02006, -0.01698752, 0.023881556, 0.009814165, -0.06561397, -0.03115648, 0.017533455, 0.006767076, 0.022573847, 0.0020885253, -0.016758988, -0.0024948036, 0.0014053109, 0.024592543, 0.011788424, 0.019209353, -0.008011304, 0.027068304, 0.023741897, -0.021786682, -0.031131087, -0.01631462, -0.0027169872, 0.012931082, -0.006475063, 0.040297747, -0.010944126, -0.023005519, 0.009255531, -0.006452845, 0.01768581, 0.03981529, 0.063379444, 0.010436278, -0.0014489541, -0.013991215, -0.01955215, -0.00057529664, -0.03105491, 0.0098712975, -0.007719291, -0.016974822, 0.004227835, -0.0047007687, -0.029632935, -0.042126, 0.026103392, 0.0074336263, 0.011896341, 0.004691247, -0.00027931645, -0.018079393, -0.03062324, 0.0119788665, 0.0062370095, -0.006513152, 0.03760615, -0.033289444, -0.031308834, 0.033441797, 0.004786468, 0.0053038383, 0.011896341, 0.0091349175, -0.014499063, 0.031537365, -0.0060370443, -0.013521455, -0.0033454492, 0.0019012563, 0.0010958409, 0.023462582, -0.039561365, 0.018269835, 0.00082445965, -0.030470885, -0.020313924, -0.042303745, 0.034178175, 0.009077785, 0.022167569, 0.0049959556, -0.009909386, 0.02577329, 4.9396163e-05, 0.011947126, -0.0060560885, -0.01082986, 0.012734291, -0.020720202, -0.016974822, 0.02581138, -0.008798468, -0.021367708, -0.045223873, -0.016974822, -0.022573847, -0.009280924, -0.0006728987, 0.03095334, 0.007471715, -0.023437189, 0.012429582, -0.008880993, 0.004691247, -0.026535062, 0.01109648, 0.029861467, 0.037047517, -0.022980126, -0.013318316, -0.0068876897, 0.024706809, 0.016593937, -0.00277412, 0.03516848, 0.004338927, -0.008360449, 0.013851557, 0.022535758, -0.017939733, -0.014181658, -0.009261879, -0.029658329, 0.050454706, -0.0015719485, -0.010950474, -0.02075829, -0.008360449, 0.00122439, -0.05860567, -0.01764772, 0.005643462, 0.010594981, 0.0031708765, 0.0006947203, -0.0030439144, 0.012048696, 0.00966181, -0.034711417, -0.010791772, -0.021964429, -0.010144265, -0.029455189, 0.017127177, -0.014435582, -0.009642766, -0.010302968, 0.007401886, -0.024948038, -0.025532063, -0.031689722, -0.016835164, 0.019679114, -0.008366797, 0.002394821, 0.007401886, 0.0043040127, -0.015743291, 0.016073393, -0.020148873, 0.03382268, -0.01206774, -0.01942519, 0.032984734, -0.0031867467, -0.05378111, 0.019171266, 0.0114011895, 0.018079393, -0.025646329, 0.01969181, 0.028083999, -0.010144265, 0.014702202, -0.009433278, 0.0123470565, 0.0051641804, -0.012683506, -0.026585847, 0.032883164, 0.03813939, 0.028464885, -0.015133873, 0.03727605, 0.0017980996, -0.017736595, 0.0014822816, -0.03222296, 0.060433924, 0.009217443, -0.010690202, -0.015006911, -0.03115648, -0.01420705, -0.013648418, -0.012328012, -0.004338927, 0.005662506, -0.03339101, -0.015184658, -0.017977823, -0.022853164, 0.040196177, 0.022053303, 0.0069829114, -0.029607544, -0.05301934, 0.024859164, -0.009668158, -0.050124608, -0.030927949, -0.0017600111, 0.019069696, 0.04210061, 0.015362404, -0.012962823, 0.011077437, -0.00939519, 5.5000346e-05, 0.0032787942, -0.010360101, 0.007979563, 0.02212948, -0.0018092088, 0.020885253, -0.005199095, 0.008062088, 0.002818557, 0.012118525, 0.04860106, 0.0030312182, -0.01627653, 0.019183962, -0.0005939442, -0.0086080255, -0.0038755157, -0.0106648095, 0.04202443, -0.014422886, -0.013026304, 0.02444019, 0.0062243133, 0.024287835, 0.019679114, 0.007731987, 0.0120867845, -0.021240747, -0.0021662896, -0.01848567, -0.0006982911, 0.004192921, -0.009737987, 0.0012053457, -0.0032311834, -0.019361708, -0.011426582, -0.022738898, 0.014232443, 0.0076367655, 0.022383405, 0.008544544, 0.02423705, 0.025519367, -0.012220095, -0.0056275916, 0.0041040475, 0.019869557, -0.0264081, 0.018371405, 0.037428405, 0.04839792, -0.0079224305, 0.0149815185, -0.018612633, -0.006014826, -0.008462018, 0.034686025, -0.006735335, -0.0026011344, -0.0146133285, -0.008747683, -2.2193557e-05, 0.001159322, 0.033441797, -0.005043566, -0.013661114, 0.0025344794, -0.00091253954, 0.025049606, 0.015844861, 0.013800772, -0.029455189, -0.0040469146, 0.020694809, 0.0035549367, 0.012359753, -0.02026314, -0.011578936, -0.053730328, -0.005272098, -0.008011304, -0.002440845, -0.014956126, 0.022459581, -0.008303316, -0.01614957, -0.00082445965, -0.008214443, -0.0031312008, 0.012670809, 0.006881342, 0.01731762, -0.035930254, -0.024033912, -0.04067863, -0.041415013, 0.011102829, -0.033416405, 0.03283238, -0.017800076, 0.027906252, 0.0056148954, 0.035650935, -0.020415492, -0.039891466, 0.02153276, 0.014740291], "content_hash": "8a1d48fce4902742bac6ee5ae8ae076d"}, {"content": "the **head categories**. You should see heads grouped into Previous-Token, First/Positional, Bag-of-Words, Syntactic, and Other. - Click on a category (like \"Previous-Token\") to see which specific heads belong to it. - Below the categories, you'll see the **BertViz visualization**. Try clicking on individual head squares to see their attention patterns.\n\n**Stage 4 - MLP**: Click to expand. You'll see the expand-compress pattern: 768 \u2192 3072 \u2192 768. This shows GPT-2's feed-forward network dimensions.\n\n**Stage 5 - Output**: Click to expand. You'll see:\n- Your prompt with the predicted next token highlighted\n- The confidence percentage\n- A top-5 bar chart showing the model's top predictions\n\n### Step 6: Reflect\n\nThink about what you observed:\n- How many tokens did your prompt become?\n- What was the model's top prediction? How confident was it?\n- Were there any surprising alternative predictions in the top 5?\n\n## What's Next?\n\nTry changing the prompt and running the analysis again. Compare results with different inputs:\n- A factual prompt: \"The capital of France is\"\n- A creative prompt: \"Once upon a time, there was a\"\n- A technical prompt: \"The function takes an input and\"\n\nThen move on to **Experiment: Exploring Attention Patterns** to dive deeper into what the attention heads are doing.", "source_file": "experiment_first_analysis.md", "chunk_index": 1, "embedding": [0.0076011117, 0.013393115, 0.0067180097, -0.023311028, -0.028459476, -0.017890858, 0.014465709, 0.060808893, 0.033808142, 0.04550656, 0.043304168, -0.010475662, -0.014658776, -0.042932335, -0.02884561, -0.008487789, -0.006646503, 0.025427613, 0.0051162704, 0.053029016, -0.019263776, -0.008130258, 0.009689094, 0.055574637, 0.0413878, 0.007493852, -0.026500206, 0.043990627, 0.01791946, 0.012856819, -0.0042796475, -0.019592704, -0.01917797, -0.005495253, -0.009481725, -0.004715835, -0.023039306, 0.022624569, -0.020136151, 0.0275728, -0.0016634135, -0.009782052, -0.029431961, 0.02585665, -0.026013963, 0.02126595, 0.024226308, -0.040958766, -0.02791603, 0.060294047, -0.029017225, 0.057548206, -0.01563126, 0.0321492, 0.00012535935, -0.0044977413, -0.032921463, -0.010904699, -0.002449088, -0.015688466, 0.023639956, -0.003997198, 0.0402151, 0.031977583, 0.002522382, -0.041416403, -0.06492765, 0.01803387, 0.019978838, 0.005884962, 0.054745164, 0.026457302, -0.0031069454, 1.6284424e-05, 0.008723759, -0.004272497, 0.03872777, 0.0060530016, 0.0050519146, -0.03998628, 0.029145936, -0.028216356, -0.010647276, -0.038641963, 0.0012853243, -0.045449354, -0.045334946, -0.05308622, -0.043275565, -0.07270753, -0.022166928, 0.016260516, -0.022581665, 0.034752022, 0.01355043, -0.006296123, 0.020136151, -0.014959102, -0.010318348, 0.0019127915, 0.0413878, 0.0012030922, -0.03160575, -0.008931127, 0.036496777, 0.0064427108, 0.013035584, 0.010068076, -0.03283566, -0.020736804, -0.10171045, 0.013078488, -0.010060926, 0.032235004, 0.034065563, 0.01943539, -0.026128374, -0.026156977, 0.045535162, -0.044791497, -0.007150622, -0.014186835, -0.005720498, 0.016875468, -0.0014935862, 0.0037183235, 0.026342893, -0.032692645, 0.011162122, -0.015173621, -0.01584578, 0.017890858, 0.039700255, -0.017104289, -0.004161662, -0.02840227, -0.011104916, 0.01505921, -0.0482524, 0.01793376, -0.0063497527, -0.007168499, 0.0024115473, 0.009009784, -0.05411591, -0.02345404, -0.027529895, 0.06498486, -0.043475784, 0.01942109, 0.01540244, 0.0055095544, -0.029832397, -0.027186666, -0.03389395, -0.037526466, -0.00142208, 0.025255999, -0.034523204, -0.052228145, 0.0367542, 0.0033089505, -0.0025885252, 0.005895688, -0.038813576, 0.04871004, -0.031233918, 0.0100251725, -0.030747676, -0.05849209, -0.04044392, -0.008316174, 0.025885252, -0.0505692, 0.020250563, 0.006271096, -0.07116299, -0.013722044, -0.012613698, -0.041616622, -0.022553062, -0.0025885252, -0.003664694, 0.00096622785, -0.044105038, -0.008173161, -0.00465148, 0.017576229, -0.024154803, -0.0079514915, -0.0042903735, 0.024226308, 0.085693054, 0.011584008, 0.0007432178, -0.046479046, 0.05723358, 0.002631429, 0.022967799, 0.0051699, 0.0195212, 0.04104457, 0.037755284, -0.021466168, 0.031548545, 0.017561927, -0.018691726, 0.014158232, -0.034666218, -0.029660782, 0.022624569, -0.038212925, 0.057061967, 0.040072087, -0.08746641, 0.0088810725, -0.07819921, 0.023754368, -0.001131586, 0.052571375, 0.048767243, 0.008080203, 0.044848703, 0.019249475, -0.012284769, 0.042560503, 0.056861747, -0.028330766, 0.017576229, -0.013543279, 0.024498032, 0.02424061, 0.0012620848, -0.008580747, -0.031205315, -0.035152458, 0.040615536, -0.026328592, 0.01883474, -0.047422927, 0.031777367, 0.023039306, 0.030347241, 0.0032195677, 0.046250224, 0.02677193, -0.015159319, -0.04318976, 0.050368983, 0.04736572, 0.048452616, -0.029346153, -0.042674914, 0.005720498, -0.0032874986, -0.02814485, -0.032320812, 0.027987536, -0.023811571, -0.0070862668, -0.000114018905, 0.012921174, -0.025885252, 0.0043261265, -0.01631772, -0.030919291, -0.046135817, -0.03469482, -0.05354386, -0.061666965, -0.054659355, -0.025842348, 0.046707865, 0.025127286, -0.052914605, -0.0023114386, 0.002034352, 0.0016669888, -0.018777534, 0.009867859, -0.022023916, -0.014830391, 0.025356106, 0.028802706, -0.014873294, 0.008802416, -0.0006288079, -0.009159947, -0.0075510573, -0.009488876, 0.036382366, -0.05094103, 0.035953328, -0.020064646, -0.010904699, -0.0020951324, 0.031062303, 0.025699336, 0.007915739, 0.018448606, 0.02494137, -0.05654712, 0.013707743, 0.0061745625, 0.010761687, 0.029489167, 0.034094166, 0.011591159, 0.005505979, -0.005541732, -0.024212006, -0.024154803, 0.030375844, -0.019206572, -0.020407876, -0.021880904, 0.027630005, 0.023911681, 0.016274815, -0.012284769, -0.038184322, -0.016904071, 0.00040445707, 0.034666218, 0.010010871, -0.017862255, -0.023482643, 0.020751106, 0.0046193018, -0.028202055, 0.038384542, -0.0026457303, -0.028688297, 0.047651745, -0.021551976, -0.009059838, -0.005330789, 0.03632516, 0.0024794783, 0.061838582, -0.08128827, 0.01586008, -0.015831478, 0.0019378186, -0.03652538, 0.07276473, 0.00362179, -0.0011280107, 0.0027369007, -0.007872835, -0.042474695, 0.011162122, 0.026013963, -0.02402609, 0.007822781, 0.04570678, -0.011905786, -0.009782052, 0.0076797684, 0.0063390266, -0.018176882, 0.009467424, 0.046135817, 0.013629086, 0.017075686, -0.005927866, -0.032435223, -0.01297838, 0.0027154488, -0.022481557, 0.0008330475, -0.006213891, 0.02009325, -0.028631091, -0.056232493, -0.015588356, -0.017604832, -0.0061280834, -0.026328592, 0.08443455, -0.007132746, -0.023396837, -0.020937022, -0.018977752, 0.022710375, 0.027715812, 0.033607926, -0.014715981, -0.05803445, 0.0149448, 0.0004924991, -0.035753112, -0.023854475, 0.003493079, -0.019578403, 0.04676507, -0.017190095, 0.04702249, 0.04230308, 0.071506225, 0.017962363, -0.016603746, -0.047880568, -0.00055149174, 0.021766495, 0.014587269, 0.007479551, 0.016003093, 0.023411138, -0.024412224, 0.049253486, -0.03735485, 0.003083706, -0.01447286, -0.009281508, 0.044333857, 0.051513083, 0.00523068, 0.02378297, 0.039671652, -0.016017394, -0.051084045, -0.020722503, -0.0015373838, -0.056261096, -0.0037183235, 0.05505979, -0.03503805, 0.037383452, -0.020979926, 0.0018135766, 0.056718737, 0.012556492, -0.0097391475, 0.017976664, -0.036468174, 0.035695907, 0.066243365, 0.026114073, -0.033035874, 0.017261602, -0.029174538, -0.019220872, -0.02652881, -0.048424013, 0.011390941, -0.03134833, -0.07791318, 0.027401185, -0.04630743, -0.01563126, 0.029045828, -0.02218123, 0.047994975, -0.026857737, -0.031920377, 0.06584293, -0.014036671, 0.07768436, -0.0050054356, 0.0012996256, 0.017490422, -0.010218239, 0.029832397, 0.0036861459, 0.032320812, -0.007207827, -0.026042566, 0.041816838, -0.0020146878, 0.012306221, 0.0005943955, 0.002091557, -0.03652538, 0.003932842, -0.022567363, -0.01183428, 0.015459646, -0.015116416, -0.055689048, 0.021537675, 0.0065428196, 0.027944632, 0.044991717, -0.0036789952, 0.07739834, -0.03941423, -0.0075081536, 0.027429787, 0.007901438, 0.008423433, -0.015330934, 0.016189009, -0.01746182, 0.033493515, -0.016618047, -0.0076225633, -0.034637615, -0.0033482788, 0.0060637277, 0.02045078, -0.022095423, -0.019206572, 0.054058705, 0.033293296, 0.011176422, 0.05205653, 0.03538128, -0.0032946493, 0.044391062, -0.07139181, -0.008952579, -0.021880904, 0.0144585585, -0.019764319, 0.01654654, 0.016832564, 0.023482643, 0.014279793, 0.020522285, -0.002361493, 0.0070469384, -0.013657688, -0.0014631961, -0.013929412, -0.013736346, -0.021780796, -0.04184544, -0.03415137, -0.034437396, 0.01896345, -0.036496777, 0.0058241817, 0.013857906, -0.02954637, 0.007400894, -0.007894287, -0.009896461, -0.04147361, 0.020393575, 0.0036360915, 0.012892572, 0.01229907, 0.019006355, -0.010196787, 0.009496027, 0.022824787, -0.0073722918, 0.016532239, 0.02734398, 0.007994396, -0.023225222, 0.010547168, -0.014687378, -0.0035878248, -0.010182486, -0.04098737, -0.034980845, 0.021694988, -0.029088732, 0.027415486, -0.01999314, 0.021551976, -0.025255999, -0.0052092285, 0.036153547, 0.029632179, -0.007400894, -0.0074294964, -0.024712551, 0.007629714, -0.0039042397, 0.027658608, -0.001889552, 0.024312116, -0.025227396, -0.01987873, 0.017719243, -0.013922262, 0.015245127, -0.022996401, -0.026700424, -0.01125508, 0.021108637, -0.006253219, -0.0060351253, 0.017032782, -0.004218867, -0.02136606, -0.015874382, 0.02311081, -0.027286775, -0.002331103, 0.034179974, -0.02170929, -0.0097391475, 0.0128139155, -0.01896345, -0.04908187, 0.008780964, 0.017504724, 0.005920715, -0.032235004, 0.0042438945, -0.0333219, 0.02459814, -0.029246045, -0.013149994, 0.02827356, 0.020650998, -0.022881992, -0.024626743, -0.03389395, -0.023153715, -0.029231744, 0.0018770384, -0.031005098, -0.002974659, 0.013200048, -0.0018430729, 0.024397923, 0.0038720618, 0.0030265008, 0.01791946, 0.008294722, 0.011619762, 0.022881992, -0.015960189, -0.0007012079, 0.009374466, -0.02585665, 0.012163209, 0.03297867, 0.007987245, -0.026457302, -0.006821694, 0.0016169344, 0.026013963, 0.016861167, -0.0028888513, 0.021094335, 0.012306221, -0.03492364, -0.019192271, 0.0018824013, -0.009567533, 0.0033232516, 0.008788114, -0.028001837, -0.0195212, -0.059207153, -0.035009447, 0.05045479, -0.006574997, 0.000981423, -0.011104916, -0.0100251725, -0.002901365, -0.016689552, 0.028530983, -0.019120764, 0.0019270927, -0.007143472, -0.05305762, -0.020164754, 0.03046165, 0.010947603, 0.02218123, 0.01907786, -0.0067537627, -0.081345476, 0.011734171, 0.025842348, 0.036239352, 0.013085639, -0.027758716, -0.0070397877, -0.039128207, -0.009960817, 0.03964305, -0.002445513, 0.014129629, -0.07402324, -0.006914652, 0.012842517, -0.0033625802, 0.011905786, -0.024312116, 0.013414567, 0.006600024, -0.0044047832, -0.035695907, 0.0021559126, -0.005995797, -0.007787028, -0.0036450296, -0.0030139873, 0.054316126, -0.0048767244, -0.054401934, -0.01114067, -0.0062925476, -0.01698988, -0.029689383, 0.01080459, -0.031119509, -0.010397005, 0.050511997, 0.022438653, 0.002170214, -0.046107214, -0.011612611, -0.011662665, 0.009274357, -0.022395749, 0.032778453, -0.014208286, -0.011126368, 0.012949777, -0.035896122, -0.024154803, 0.013300157, 0.002652881, 0.010232541, 0.021165842, -0.02253876, -0.016575143, -0.018706027, -0.001489117, 0.02793033, 0.028101945, 0.03492364, 0.0063819303, -0.056518517, -0.0064784638, 0.023196619, -0.026500206, 0.01194869, -0.025327504, -0.039357025, 0.020822613, -0.002937118, -0.006582148, 0.01114067, 0.034780625, -0.0071291705, -0.020307768, 0.035552893, -0.02309651, -0.044190846, -0.021294553, 0.014079575, -0.014858993, -0.016289117, 0.011855732, -0.016031696, 0.014558667, -0.020779708, 0.008115957, 0.033121683, 0.012456384, 0.002080831, -0.02010755, 0.013757797, 6.659017e-05, -0.03572451, 0.02619988, 0.013571881, 0.009445972, 0.008194613, 0.016475033, -0.016875468, 0.034609012, -0.017776446, 0.063955165, -0.028101945, 0.03918541, -0.0022971374, -0.012821066, 0.020236261, -0.0095961355, 0.00862365, -0.031920377, -0.04307535, 0.011362338, -0.009066989, -0.016346322, -0.021394663, -0.013593333, -0.020393575, -0.018577317, -0.040872958, 0.0043976326, -0.04307535, -0.0019682087, -0.010654427, -0.008602198, -0.10811741, -0.011912936, 0.016946975, -0.03735485, -0.024140501, -0.013836454, 0.0321778, -0.010218239, -0.012999832, 0.0057276483, 0.018577317, -0.017447518, -0.00269936, -0.022424351, 0.0028566737, -0.044190846, 0.0060637277, -0.008258969, 0.001224544, -0.040815752, -0.019263776, 0.010497114, -0.0018055321, 0.023025004, -0.03735485, 0.0055381567, -0.046021406, -0.007965793, 0.02804474, 0.015245127, -0.0356387, 0.02378297, -0.0034215727, 0.016231913, 0.0117413215, 0.013285856, -0.025627831, 0.007987245, 0.017418915, -0.012899723, 0.010053775, -0.015216524, 0.0100251725, -0.035238266, 0.04070134, 0.0037612272, -0.019249475, -0.011626911, 0.011090615, 0.033264697, -0.0070612393, 0.0072757583, 0.011498201, 0.0012844305, 0.04113038, 0.014286944, -0.008502089, -0.021509072, 0.026271386, 0.044162244, -0.005309337, 0.01698988, 0.040586933, -0.015316633, -0.009567533, 0.0047051096, 0.007214978, 0.030204229, -0.042131465, -0.01056862, -0.027086558, 0.0029263922, -0.0006180819, -0.007844233, -0.0043976326, -0.009038387, 0.009045538, 0.0033661555, 0.012577944, -0.012785313, 0.058635104, -0.025527721, -0.018620221, 0.021080034, 0.020994227, 0.025441915, 0.02022196, -0.038127117, 0.019792922, 0.033064477, -0.00054881023, -0.018219786, -0.016646648, 0.016946975, -0.020708201, -0.005920715, 0.018934848, 0.03046165, 0.038927987, 0.0014631961, 0.03658258, 0.042102862, -0.0038112816, -0.038184322, -0.053515255, 0.020379273, 0.0004924991, 0.006685832, -0.006149535, -0.011634062, 0.024040392, 0.0058670854, 0.029231744, 0.03363653, -0.00025876315, 0.04953951, 0.05137007, -0.006835995, 0.0006766276, -0.034895036, 0.010032323, 0.022896292, -0.036039136, 0.00591714, -0.025627831, 0.008502089, -0.03011842, 0.006986158, -0.0052342555, -0.017561927, -0.001411354, 0.020851215, -0.022009615, -0.005766977, -0.011941539, -0.0044762897, -0.007286484, 0.01769064, -0.012263318, 0.016060298, -0.0039757458, 0.008030149, 0.031090906, 0.015373838, -0.012914023, 0.005212804, -0.0080945045, -0.017047083, 0.0014033096, 0.025913855, 0.02837367, 0.0043797563, 0.010976206, -0.0057169227, 0.00500186, 0.034551807, 0.025842348, -0.0048767244, -0.011412393, 0.02804474, 0.0019127915, 0.042960938, 0.042503297, -0.020493684, 0.0035270446, -0.018462908, -0.022839088, 0.005352241, 0.0023471918, 0.046707865, -0.0540015, 0.028745502, 0.04113038, 0.01631772, 0.022224134, 0.043933425, 0.010389854, 0.02022196, 0.024054693, 0.02665752, -0.012127455, 0.032006186, -0.020364972, 0.0010377341, 0.020951323, -0.0053343643, -0.009124194, 0.021666385, -0.007218553, -0.021051431, -0.009074139, 0.027901728, -0.006278246, 0.047994975, -0.026714725, -0.007779877, 0.017619133, -0.003108733, -0.018877642, -0.005277159, -0.027057955, 0.030032614, 0.00873091, -0.013514676, 0.0027333254, 0.021008529, -0.01034695, -0.01780505, 0.0062460685, 0.007011185, -0.019263776, 0.010940452, -0.030347241, 0.007400894, -0.033150285, -8.3517036e-05, 0.02045078, 0.008702307, 0.023010703, -0.0063962317, -0.04848122, -0.022839088, -0.030719073, 0.021852301, -0.0053379396, -0.009181399, 0.038756374, -0.014630173, 0.00943167, 0.00442266, -0.01079744, -0.007168499, -0.009438821, -0.0044441116, -0.02712946, -0.0014086725, 6.340591e-05, -0.0066035995, 0.005101969, -0.0014524702, 0.038241528, 0.022038218, 0.012484986, 0.005523856, 0.017776446, -0.020636696, 0.0147445835, -0.0029889601, -0.006203165, -0.018920546, -0.014558667, 0.011205025, 0.019063558, -0.016031696, 0.0020540163, 2.360264e-05, 0.039471436, 0.016975578, -0.00036155334, 0.013807852, -0.006414108, 0.027444089, 0.0011986231, -0.0574338, 0.016975578, -0.0043833316, -0.020121852, -0.013049886, 0.023225222, -0.0076225633, -0.0014766034, 0.037269045, -0.02092272, 0.033836745, 0.0031623626, 0.00746525, 0.007665467, -0.015559754, 0.019592704, -0.006871748, 0.0128854215, 0.0037719533, 0.008216065, -0.011362338, 0.0128854215, 0.023468342, -0.01884904, -0.02010755, 0.006410533, -0.027158063, 0.025413312, -0.013192898, 0.0015579418, 0.011469598, -0.001941394, -0.006861022, 0.0051412974, -0.0028977897, 0.008952579, -0.06006523, 0.036239352, 0.007665467, 0.021795096, -0.02193811, -0.00586351, 0.006986158, 0.03606774, -0.048281003, 0.012141757, -0.004926779, -0.014158232, 0.010840343, -0.005505979, 0.022267038, -0.01171272, -0.010904699, 0.015445344, -0.03572451, 0.0044298107, -0.013922262, -0.02780162, 0.0091670975, -0.02884561, 0.023010703, 0.0033822444, -0.027944632, 0.008344776, -0.011676966, 0.01698988, -0.003092644, 0.010783139, -0.0004889238, -0.010325499, 0.011662665, -0.038584758, 0.0028012563, -0.028859911, -0.006800242, 0.006499916, -0.006378355, 0.0026385796, 0.033836745, 0.014916198, -0.011584008, 0.015802875, -0.0041545117, 0.018234087, -0.007593961, -0.003925692, 0.00884532, 0.069504045, -0.030661868, -0.026156977, -0.009867859, 0.00873091, 0.020522285, -0.005320063, 0.0072686076, -0.0413878, -0.0036611187, -0.018133977, 0.0029442688, -0.019707115, -0.0055202805, 0.023711463, -0.014494311, -0.007829932, -0.012070251, 0.06424119, -0.0026439426, 0.020722503, 0.04032951, -0.019764319, -0.0025348957, 0.006178138, 0.04619302, 0.016475033, 0.011340887, 0.00037965336, 0.0071005677, 0.0002828965, -0.03069047, -0.006921802, 0.014758884, -0.021308854, 0.0051198457, 0.007851384, 0.043504387, -0.0066035995, 0.014758884, -0.012291919, 0.00815886, 0.03403696, -0.021966713, 0.013335911, 0.023411138, 0.0051520234, 0.015116416, -0.017704941, -0.025556324, -0.026257085, 0.025441915, 0.00054121274, -0.012234715, 0.029174538, -0.04344718, -0.019063558, 0.009338712, 0.004065129, 0.0069933087, -0.023768669, 0.02918884, -0.016117502, 0.013772098, -0.0013532552, -0.056918953, -0.019792922, -0.006596449, -0.009989419, 0.026829135, -0.028244957, -0.024140501, -0.0061423844, -0.01666095, 0.02469825, -0.0033089505, 0.026986448, -0.0028298588, 0.030318638, 0.013714894, -0.011548255, -0.010618674, -0.04173103, -0.0161032, -0.006986158, 0.0038899386, 0.039128207, -0.023196619, -0.021537675, 0.017290205, -0.021737892, 0.022910593, 0.013185748, 0.04756594, -0.0029156662, -0.0041509364, 0.003979321, -0.015416741, -0.0041223336, -0.036639787, -0.013521827, -0.017290205, -0.009674792, 0.009345863, -0.01068303, -0.03526687, -0.058349077, 0.0073865927, 0.006835995, 0.001582969, 0.01447286, -0.006507066, -0.002896002, -0.017776446, 0.007722672, -0.0034376616, -0.020851215, 0.05620389, -0.049139075, -0.033951156, 0.029489167, -0.010146733, -0.003925692, -0.0033822444, 0.012284769, 0.009567533, 0.018977752, -0.006768064, -0.03712603, -0.008802416, -0.016332021, 0.029775191, 0.018091075, -0.0039828965, 0.0004947337, 0.0097391475, -0.0010967267, -0.025756542, -0.043990627, 0.041015968, 0.0037433507, 0.025370408, -0.010182486, -0.02608547, 0.016003093, -0.0073579904, 0.019907333, 0.013621936, 0.004218867, 0.014043822, -0.028445175, -0.030604664, 0.02517019, 0.018434305, 0.0041795387, -0.035238266, -0.000581435, -0.0068252687, -0.00471226, -0.018319894, 0.030604664, -0.016932674, -0.021652084, -0.004898176, 0.012020196, 0.023625655, -0.019764319, -0.0013675565, 0.021809397, 0.01229907, -0.00024111004, -0.037612274, -0.027973235, 0.02701505, 0.009496027, -0.011047712, -0.0002609977, 0.045535162, -0.02355415, 0.017490422, 0.0033161012, -0.002407972, -0.026142675, -0.03249243, -0.0039471434, 0.020879816, -0.003544921, -0.016403528, -0.019392489, 0.0029978985, 0.00045652254, -0.039786063, -0.0128854215, 0.008230366, 0.023225222, 0.059950817, -0.018133977, 0.028445175, 0.03134833, -0.017833652, -0.014565818, -0.019792922, -0.027386883, -0.019621307, -0.0275728, 0.037040222, -0.034780625, -0.016045997, 0.009231454, 0.0059922216, -0.028530983, -0.009102742, -0.037240442, -0.010697331, 0.0153881395, -0.007915739, 0.013142844, 0.007980094, -0.004354729, -0.027200967, 0.016460732, -0.018520111, 0.042817928, 0.0027065105, -0.0137864, 0.011197874, -0.0009483513, -0.03929982, 0.008144558, -0.0018019568, 0.030404447, -0.037755284, -0.017304506, 0.023482643, 0.015473947, 0.010654427, -0.02619988, 0.015817177, 0.013293006, 0.015574056, -0.0052807345, 0.045763984, -0.01643213, 0.01884904, -0.01781935, 0.03887078, -0.0108117405, 0.01781935, -0.023039306, -0.011047712, 0.028859911, 0.012935475, -0.03203479, -0.027444089, -0.00918855, -0.040815752, 0.0033125258, 0.0138150025, -0.014773185, 0.0064212587, -0.060179636, -0.014830391, 4.0306048e-05, -0.0003237891, 0.015002006, 0.029632179, 0.0047515887, -0.022710375, -0.04150221, 0.011348038, -0.020608094, 0.0038470349, -0.023339631, -0.010754536, 0.001152144, 0.051799107, -0.001370238, 0.00059082016, 0.004358304, -0.020465082, 0.010268293, -0.031891774, -0.0063497527, 0.002854886, -0.019950235, 0.008630801, 0.0007432178, 0.0107402345, 0.031233918, -0.014673077, 0.029775191, 0.028259259, -0.015931586, -0.021323156, -0.00050054357, 0.0013916899, -0.0034483876, -0.024512334, -0.025685035, 0.019149367, -0.041988455, -0.011469598, 0.012992681, -0.0046479045, 0.035152458, 0.011047712, 0.009510328, 0.020879816, -0.019893032, -0.012713807, 0.020879816, -0.008173161, 0.012463534, -0.008180312, 0.0080516, -0.0141725335, -0.008659404, -0.014201135, -0.016632348, 0.01631772, -0.006410533, 0.016789662, 0.026485905, 0.017533327, 0.022810485, -0.008530692, -0.005266433, 0.0036003382, 0.030433048, -0.024512334, 0.008645102, 0.040529728, 0.034666218, 0.010411306, 0.011455297, -0.0016991666, -0.02010755, 0.017747844, 0.03229221, -0.00057070906, -0.022023916, -0.015273729, -0.010933301, -0.032320812, -0.0084520355, 0.039271217, -0.022424351, -0.033121683, -0.010304047, 0.014844691, 0.0011852157, 0.019278077, -0.028631091, -0.019978838, -0.0020218384, 0.016231913, 0.018148279, 0.033407707, -0.024498032, -0.030747676, -0.060522866, -0.02585665, -0.009639039, 0.0067072837, -0.0206939, 0.019363886, -0.046793673, -0.044162244, -0.028874213, -0.011848581, -0.017676339, 0.0063283006, 0.030261433, 0.0008688006, -0.02894572, -0.028759804, -0.040872958, -0.0402151, 0.021037132, -0.030232832, 0.028645393, -0.021280251, 0.020364972, -0.006306849, 0.013085639, -0.010139583, -0.008716608, 0.016060298, -0.023539849], "content_hash": "c2532e2b268dc04c99cd84078753518c"}, {"content": "# Experiment: Understanding Token Attribution\n\n## Goal\n\nLearn how to use token attribution to identify which parts of your input most influenced the model's prediction. Compare two attribution methods and see how results change with different target tokens.\n\n## Prerequisites\n\n- Complete \"Your First Analysis\"\n\n## Steps\n\n### Step 1: Run an Analysis with a Meaningful Prompt\n\n1. Select **GPT-2 (124M)** and enter the prompt: `The capital of France is`\n2. Click **Analyze**.\n3. Check Stage 5 -- the model should predict something like \"Paris\" or \"the\" with high confidence. Note the top prediction.\n\n### Step 2: Open the Attribution Panel\n\n1. Scroll down to the **Investigation Panel**.\n2. Click the **\"Token Attribution\"** tab.\n\n### Step 3: Run Simple Gradient Attribution\n\n1. Select **\"Simple Gradient (faster, less accurate)\"** as the attribution method.\n2. Leave the **Target Token** dropdown empty (this defaults to the top prediction).\n3. Click **\"Compute Attribution\"**.\n\n### Step 4: Read the Results\n\nLook at the two visualizations:\n\n**Color-coded tokens**: Your input tokens are displayed as colored boxes. Darker blue means higher influence.\n- You should see that **\"France\"** has a very dark color -- it's the most relevant token for predicting \"Paris.\"\n- **\"capital\"** likely also has a notable color -- it sets up the context for a city name.\n- Function words like **\"The\"**, **\"of\"**, and **\"is\"** should be lighter -- they contribute less to this specific prediction.\n\n**Bar chart**: Shows the same information as horizontal bars with scores. Longer bars = more influence.\n\n**Hover over any token chip** to see the exact attribution score.\n\n### Step 5: Compare with Integrated Gradients", "source_file": "experiment_token_attribution.md", "chunk_index": 0, "embedding": [-0.013251405, -0.00042796275, -0.017115343, -0.035826936, 0.017482733, -0.016367894, -0.0051561394, 0.034230687, 0.023867732, 0.029923346, 0.021802742, -0.00716412, -0.025970727, -0.059694666, -0.02660416, -0.00016756266, -0.009780196, 0.014670294, 0.007689869, 0.07003228, -0.04466965, -0.010476972, 0.03972888, 0.041249115, 0.032279715, -0.043808185, -0.021397347, -0.006156963, -0.013846832, 0.025413306, 0.020092474, -0.010717676, -0.046975344, -0.035548225, -0.014188886, -0.030379416, -0.038183305, -0.007189458, 0.015506425, 0.00067579316, -0.0063311565, -0.0056597185, -0.038411338, 0.02847912, -0.026908208, 0.01565845, 0.0064863474, -0.02820041, -0.031266224, 0.05396844, -0.028783167, 0.034458723, -0.015709123, -0.00022170134, -0.045303084, 0.030404754, -0.015493756, 0.005381008, -0.03466142, -0.03785392, -0.03792993, -0.027769675, 0.06623169, 0.009944889, 0.03418001, -0.02979666, -0.039272808, 0.00052772835, 0.0070184306, -0.0032748454, 0.041705187, 0.016139857, 0.022398168, -0.0031497425, -0.04720338, -0.024513833, -0.023639696, 0.05199213, -0.026021402, -0.039576855, 0.011515801, -0.016621266, -0.01072401, 0.0018417047, 0.008836382, -0.02192943, -0.028327096, -0.034813445, -0.03501614, -0.06420471, 0.0033381886, 0.00494394, -0.025248615, 0.014771643, 0.041730527, 0.0005415847, -0.023500342, -0.027237592, 0.013276743, 0.0047475756, 0.03830999, -0.01948438, 0.014568944, -0.0064040013, 0.015519094, -0.021954766, 0.04104642, -0.0032653438, -0.020092474, -0.023829727, -0.029644636, 0.050370544, -0.034306698, 0.028403109, -0.0036992452, -0.026021402, -0.05619812, 0.02885918, 0.052144155, -0.065471575, -0.025413306, -0.037803244, -0.006473679, 0.020168487, 0.015810473, -0.01255463, 0.016633935, -0.028555132, -0.00011956036, -0.027009556, -0.027085569, 0.01807816, -0.0130487075, -0.006575028, 0.002346867, -0.036789753, 0.0053430023, 0.034990806, -0.06466078, 0.019154996, -0.0118388515, -0.04983846, -0.0065560252, 0.009495151, -0.066941135, -0.017774113, 0.00590359, 0.049990483, -0.045911178, 0.03483878, 0.01166149, -0.00946348, -0.005596375, -0.030278068, -0.042389296, -0.04563247, 0.026908208, 0.017875463, 0.006001772, -0.050699927, 0.0055837063, 0.045911178, -0.0002694067, -0.017672764, -0.043225426, 0.025869379, -0.037524533, 0.004263, -0.025552662, -0.053816415, -0.025780698, -0.032862473, 0.025818704, -0.047760803, 0.03278646, -0.021118635, -0.07423828, -0.028453782, 0.0039589526, -0.03955152, -0.021992773, 0.011028058, -2.9246761e-05, -0.014011525, -0.022309488, -0.031443585, -0.007189458, 0.0064610103, 0.02885918, 0.0014188886, 0.046164554, 0.014885661, 0.060302764, 0.0072528007, -0.057769034, -0.024336472, 0.05807308, 0.02042186, 0.02660416, 0.002448216, 0.009970225, 0.0053303335, 0.030430092, -0.026224101, 0.0021410014, 0.0063659954, -0.006701715, -0.027795013, -0.06319121, -0.020928606, 0.01733071, -0.055336654, 0.022702217, -0.01349211, -0.078241564, -0.011401783, -0.0432761, 0.016380562, 0.0678026, 0.049863797, 0.017634757, 0.021219986, 0.022980927, -0.022334825, -0.032558426, 0.02137201, 0.04299739, -0.041654512, 0.027009556, -0.04241463, 0.023766384, 0.040489, -0.011541138, -0.031874318, 0.015430413, -0.06709316, 0.027617652, -0.023462335, 0.027136244, -0.033521242, 0.015139034, 0.022626204, 0.037803244, -0.012459615, 0.012586301, -0.0074681677, -0.019205669, -0.043428123, 0.025375301, 0.022398168, 0.028833842, -0.0068157325, -0.043428123, 0.029137889, 0.004456197, 0.011097736, 0.005944763, 0.010736679, 0.011541138, -0.027566977, 0.014657625, -0.0149870105, -0.033571918, -0.017381385, -0.012491287, -0.01404953, -0.047507428, 0.021308666, -0.046620622, -0.022968259, 0.0016176279, -0.01872426, 0.019750422, 0.00070231815, -0.032735787, -0.010622661, -0.033115845, 0.013479441, -0.025996065, -0.008076263, -0.01349211, -0.0030103873, 0.05640082, 0.028225748, -0.055488676, 0.0048235874, 0.030810151, 0.037245825, -0.0023246969, -0.009425473, 0.0066637085, -0.04955975, 0.034889456, -0.029644636, -0.017254699, 0.023842394, 0.017216692, 0.000498828, -0.025907384, 0.002199594, 0.011984541, -0.032457076, 0.008367642, -0.015417744, 0.028428445, 0.0062488103, 0.0099005485, 0.03501614, 0.033622593, 0.005422181, -0.04355481, -0.02575536, 0.0043136748, 0.021308666, -0.023855064, 0.06455943, 0.040894393, -0.010128584, 0.014214222, 6.987551e-05, 0.0039811227, -0.05731296, 0.022360163, 0.01750807, 0.00038936298, 0.0040729702, -0.02802305, -0.0041996567, 0.00569139, 0.015240383, -0.008792042, 0.013732814, 0.004107809, 0.038740724, -0.062228397, 0.00547919, -0.015164372, 0.033039834, 0.038259316, 0.047532767, -0.06440741, -0.018749598, 0.023411661, -0.022056116, -0.0050421217, -0.0019715582, 0.019357694, 0.017862793, 0.015886484, -0.031342234, -0.01930702, 0.0040983073, -0.010787354, -0.024032425, 0.019915113, 0.032634437, -0.009710519, -0.007056437, 0.028631143, -0.013948181, -0.02754164, 0.032431737, 0.04542977, -0.0009746941, -0.0036929108, -0.01423956, 0.000566922, 0.018344201, 0.0020079806, -0.018204847, -0.025045916, 0.023804389, -0.026781522, -0.023994418, -0.029492611, 0.008697026, 0.0025004742, 0.009919551, -0.037803244, 0.008272627, 0.027313605, -0.011439789, 0.012769997, -0.03017672, 0.04710203, 0.005026286, 0.02192943, -0.011667824, -0.026097415, 0.025616005, -0.024982573, -0.038005944, -0.01724203, 0.061569627, -0.030556777, 0.04373217, -0.0033666932, 0.03792993, 0.055133954, 0.055995423, -0.047076695, -0.015493756, -0.049129017, -0.025185272, 0.04292138, 0.066637084, 0.0040349644, 0.021029955, 0.02632545, 0.04879963, 0.036587056, -0.03744852, 0.043985546, -0.044897687, 0.012966361, 0.041097093, 0.037701894, 0.00735415, 0.035826936, 0.018964965, 0.026072077, -0.03775257, 0.008754035, -0.059339944, -0.02931525, 0.0022090955, 0.027795013, -0.013834164, 0.009913216, -0.04390953, 0.019826433, 0.009406471, 0.067245185, -0.019421037, -0.010426297, -0.0048425905, 0.05181477, 0.036587056, 0.0068410696, -0.016253876, 0.020485204, -0.03296382, -0.030683465, -0.047836814, -0.049230363, -0.014949004, -0.027972374, -0.06496482, 0.023956412, 0.017292704, -0.026300112, 0.014454927, 0.004810919, 0.034332037, -0.016013172, -0.025996065, 0.013137388, 0.033394556, 0.07788684, -0.019598398, 0.04451763, -0.002943877, -0.018597575, 0.020130482, 0.030100707, -0.0021188313, -0.001859124, -0.027693665, 0.015886484, -0.022613537, 0.018876284, -0.0057800706, -0.021600043, -0.029568624, -0.0036644062, -0.023589022, 0.022512186, 0.008836382, -0.006432506, -0.027313605, -0.0042566657, 0.029568624, 0.00993222, 0.007816556, -0.043301437, 0.0626338, 0.00018745639, 0.010217264, -0.015899153, -0.013251405, -0.00065085175, -0.053461693, 0.01657059, -0.013378092, 0.0027364278, -0.029391263, 0.020041801, -0.021511363, 0.014302903, -0.00066074915, 0.030506104, -0.055691376, -0.040260963, 0.075099744, 0.00946348, 0.005833912, 0.05189078, 0.014530939, -0.030835489, 0.0556407, -0.049889136, -0.032609098, -0.018280858, -0.011889526, -0.020510541, 0.021093298, -0.013846832, 0.047456753, 0.032761123, 0.021714061, 0.0057103927, 0.026072077, -0.015379738, -0.013618796, 0.0037594212, -0.009260781, -0.014796981, -0.03007537, -0.038157966, -0.02979666, 0.025185272, -0.022258814, 0.0014861907, 0.017394053, -0.041730527, -0.011826183, 0.016089182, -0.012801669, -0.058985222, 0.012446946, 0.035750926, -0.01825552, 0.0026255772, 0.011826183, 0.00843732, -0.011636153, 0.00018993074, -0.028403109, -0.005168808, 0.019978458, 0.01053398, -0.0402863, 0.02512193, -0.018154172, -0.009286119, -0.023500342, 0.009710519, -0.040412985, 0.003042059, -0.05219483, 0.0215367, 0.0025764862, 0.018242853, -0.020662565, -0.009938554, 0.045404434, 0.054829907, -0.014796981, 0.027896361, -9.10559e-05, 0.025907384, 0.05064925, 0.018610243, 0.006727052, 0.045303084, 0.005444351, -0.019345025, -0.005634381, 0.019623736, 0.009469814, -0.017571414, 0.00035947288, -0.004620889, 0.0059954375, -0.022461513, -0.03810729, 0.010850697, -0.001760942, -0.013390761, -0.008095266, 0.014467596, -0.011186416, 0.003268511, 0.017672764, -0.023183625, 0.021942098, 0.024374478, -0.012947358, -0.017976811, 0.009431808, 0.032634437, -0.027060231, -0.03334388, -0.007024765, -0.004183821, 0.04064102, -0.018344201, 0.025324626, -0.0072274636, 0.0050452887, -0.013099382, -0.047634114, -0.014138211, 0.012630641, -0.01377082, 0.014594282, -0.01873693, 0.01157281, 0.005358838, -0.008133272, 0.017280035, -0.013036039, 0.007335147, -0.022220807, 0.015303726, 0.045835167, -0.00086780236, 0.011585479, 0.0494584, 0.012636976, -0.02931525, 0.03298916, 0.033470567, 0.0067777266, 0.021219986, 0.0006073033, 0.004076137, 0.020434529, -0.011287765, -0.0064483415, 0.019750422, 0.008880722, -0.001039621, -0.021004617, -0.0021584209, 0.0021774238, 0.022980927, 0.04580983, -0.019421037, -0.024615182, -0.04071703, -0.017596751, 0.04740608, 0.0010934627, 0.0048615937, -0.010103246, 0.03643503, 0.0006488723, -0.028783167, -0.010888702, -0.04403622, -0.011129407, -0.046671297, -0.0074048247, -0.030784814, 0.006644706, -0.008912394, 0.032178365, 0.0264268, -0.008120603, -0.10479505, 0.010185593, -0.008887056, 0.022258814, -0.0037024124, -0.00620447, 0.0019493882, -0.04373217, -0.011091401, 0.021435352, 0.010236267, 0.0059796018, -0.04104642, -0.012763662, -0.0057579004, -0.02360169, 0.015797803, 0.0051751425, 0.013618796, -0.04515106, -0.008912394, -0.027693665, -0.0019588897, -0.005108632, -0.02529929, 0.014404252, 0.026629498, 0.04991447, -0.00035828518, -0.08234621, 0.0032415902, -0.017128011, -0.02612275, -0.013745483, -0.006746055, 0.01152847, -0.020193825, 0.024488496, 0.026401462, -0.05553935, -0.038436677, -0.0045892177, 0.030556777, 0.011186416, -0.022714885, 0.022372833, -0.0013808826, 0.016532585, -0.008887056, -0.0588332, -0.0415785, 0.011097736, -0.04525241, 0.029619299, 0.02351301, -0.0149870105, -0.026528148, 0.01873693, -0.008031922, 0.031266224, 0.0007917905, -0.010895037, -0.015721792, -0.03734717, 0.012820671, 0.01166149, -0.02014315, 0.02594539, -0.009064417, -0.005225817, 0.025324626, 0.011325771, -0.006752389, -0.030582115, 0.033951975, 0.02491923, -0.032178365, 0.026629498, -0.0164059, -0.048748955, -0.0043675164, 0.009856208, 0.022081453, -0.025147265, 0.01611452, 0.012231579, 0.014695631, -0.046417926, -0.023804389, 0.01053398, 0.0044118566, 0.034256022, -0.014036861, 0.012104893, -0.024944568, -0.019889778, 0.049787786, 0.022980927, -0.021866085, 0.0479635, -0.013074044, -0.0037214153, 0.03456007, -0.01816684, 0.044390943, -0.014036861, 0.035066817, -0.0015400324, -0.021600043, 0.005168808, 0.02275289, 0.0047222385, -0.03174763, -0.0050009484, 0.020561215, 0.0015384489, 0.009868876, -0.022917584, -0.013226069, -0.011015389, -0.017850125, -0.017634757, -0.00015044019, -0.005833912, -0.008526, 0.028783167, -0.008868054, -0.08457589, -0.018534232, 0.044289593, -0.014226891, -0.009400137, 0.0020000627, 0.028453782, 0.0068284012, -0.043960206, -0.009754859, -0.0070691053, -0.023551015, -0.0053493367, -0.018749598, -0.016621266, 0.0015344899, -0.012858678, -0.0061284583, -0.0028377771, -0.021511363, -0.014036861, 0.0014030527, 0.017837455, -0.016063845, -0.043301437, -0.006689046, -0.030987512, 0.0070057623, 0.021625381, 0.014214222, -0.012953692, 0.022930251, 0.012301257, -0.019066315, 0.014024193, 0.034154676, 0.0030705635, 0.025311958, 0.014999679, -0.026072077, 0.016190533, -0.019370362, 0.022942921, -0.026477473, 0.09243045, -0.042946715, -0.02809906, -0.011408118, -0.018914292, 0.024057763, 0.0023341982, 0.02528662, 0.021486027, -0.0048172534, 0.03326787, 0.024133774, 0.029898008, -0.037524533, -0.007259135, 0.024323804, -0.012098558, 0.034154676, -0.0022011776, 0.012928355, -0.009691515, 0.023272306, -0.007651863, 0.016152525, -0.035902947, -0.03456007, -0.02015582, -0.00037590254, 0.011142076, 0.0014030527, -0.045125723, -0.0040919734, 0.0130487075, -0.0014378915, -0.056704868, 0.014252229, 0.056907564, -0.0065053506, 0.0043706833, 0.048292886, -0.008551337, 0.03448406, 0.01743206, -0.013555453, -0.013606127, 0.03990624, 0.037727233, -0.017964143, 0.0069107474, 0.012801669, -0.025413306, -0.015557099, 0.007702538, 0.0462659, 0.018952297, -0.017444728, -0.012719322, 0.029669972, -0.06192435, -0.01138278, -0.009596501, 0.002221764, -0.0019288016, -0.032685112, -0.01825552, -0.009286119, 0.027617652, 0.031190211, -0.028403109, 0.008158609, -0.027744338, 0.006096787, 0.026958883, 0.013948181, 0.017571414, -0.012117562, 0.018850947, 0.0051783095, 0.0052543217, 6.710424e-05, -0.026629498, 0.023576353, -0.030227393, 0.0092354445, 0.010134918, 0.0055330317, 0.031823643, 0.018496225, -0.01287768, -0.036308344, -0.021676056, 0.012541961, -0.010495975, -0.013124719, -0.03473743, 0.011940201, -0.010014566, 0.01835687, 0.02135934, 0.02585671, -0.052904274, -0.014074868, -0.033571918, -0.042591993, 0.0202445, 0.026477473, 0.013365423, 0.027566977, 0.0073414813, 0.00048813882, -0.024133774, 0.008519665, 0.017229361, -0.008228287, -0.0010902955, 0.035928287, -0.009672512, 0.039779555, 0.007550514, 0.0060176076, 0.05553935, -0.030962175, 0.007430162, -0.0037625884, 0.0066827117, -0.004791916, -0.03542154, 0.011028058, 0.019927783, 0.030987512, 0.012155567, 0.022638874, 0.0016896809, 0.004301006, 0.0039589526, 0.04532842, -0.0032067515, 0.0145056015, -0.005393677, 0.012047884, 0.008215618, -0.04928104, -0.012801669, -0.005770569, -0.026401462, 0.02481788, -0.016849302, 0.032431737, 0.016279213, 0.022448843, -0.032735787, 0.0035535556, 0.024235124, 0.00044181908, -0.0019129658, -0.00311332, -0.046797983, -0.00019834352, 0.024767207, -0.010603658, 0.03744852, 0.030202055, -0.018027486, 0.008906059, 0.01724203, -0.009957557, 0.013707477, 0.005561536, -0.040894393, 0.02969531, -0.012592636, -0.0071514514, -0.03889275, 0.013327417, -0.016735284, -0.0019335523, -0.070387006, -0.025375301, -0.0093557965, 0.0013832579, -0.008982072, -0.023538347, 0.041527826, -0.03734717, 0.02313295, -0.007094443, -0.027034894, -0.020079806, -0.022778228, -0.0077342093, -0.0324064, 0.015797803, -0.0088743875, -0.0108696995, 0.023576353, 0.008354973, 0.032558426, 0.0108126905, 0.004224994, -0.004576549, -0.0013017035, -0.04074237, 0.02146069, 0.026452137, -0.011705831, 0.040869057, 0.02210679, 0.07591054, 0.02247418, -0.041857213, 0.022030778, -0.009735855, 0.048850305, 0.02512193, 0.021600043, -0.00095252396, 0.00622664, 0.035269514, 0.011110404, -0.026958883, 0.0012431111, -0.029213902, -0.01976309, 0.009210107, 0.009875211, 0.029948683, 0.00088442996, 0.024374478, -0.0016611764, 0.030810151, -0.0005613795, -0.021954766, 0.0049946145, -0.03164628, -0.0027427622, -0.007550514, -0.023158288, 0.026148088, -0.022486849, 0.016139857, 0.005225817, 0.009849873, -0.00039391578, -0.0067903954, -0.023652365, 0.004066636, 0.0018923793, -0.007233798, 0.012985364, 0.01602584, 0.014366247, 0.0138595, 0.020117812, 0.0047855815, -0.010755681, -0.010394625, 0.030506104, -0.01863558, 0.0010514978, -0.014682963, 0.005621712, 0.049965147, 0.022195471, -0.015683785, 0.010103246, -0.010673336, -0.028403109, -0.008177612, -0.019509718, 0.030379416, 0.0005736522, -0.016823964, -0.021029955, -0.038411338, -0.0035092153, -0.025134597, 0.0043928535, -0.0051941457, 0.0025147265, 0.04066636, 0.011826183, -0.0065623596, 0.0055900407, 0.013188062, -0.010850697, -0.006236142, -0.0069677564, -0.0068727415, 0.0096598435, 0.00016597907, -0.027769675, 0.01855957, -0.0132134, -0.026198763, 0.0020174822, -0.050269194, 0.021676056, 0.008443654, 0.0057452316, -0.00805726, 0.033293206, 0.010698673, 0.003813263, 0.0011663074, 0.03129156, 0.06116423, 0.065167524, -0.024969904, 0.012953692, 0.0068727415, 0.004063469, 0.038563363, -0.019839102, 0.014024193, -0.029061878, 0.023804389, 0.0076138573, 0.023855064, 0.0014093871, -0.014543608, -0.012117562, 0.024222454, 0.0037309167, 0.009444477, 0.044441614, 0.035928287, -0.008468991, 0.048419572, -0.007981248, -0.0015384489, 0.021777404, 0.015506425, 0.009704184, 0.016165195, 0.018521562, 0.022448843, 0.012674982, -0.020687902, 0.01657059, 0.0019256345, -0.025768029, 0.025083922, -0.005577372, 0.026933545, -0.03491479, 0.0008606762, 0.005415847, 0.008076263, 0.02276556, 0.0006734178, -0.0030452262, 0.016507247, 0.001583581, -0.027972374, -0.010229933, -0.027338943, -0.018420214, -0.005441184, -0.032735787, -0.012915687, 0.0144422585, -0.001304079, 0.0022550193, -0.020016463, 0.0003188936, -0.022930251, 0.005203647, 0.02078925, -0.034104, 0.013884838, 0.022499518, -0.05039588, -0.014657625, -0.0026065742, 0.0033571916, 0.007550514, -0.020409191, -0.0068980786, -0.042110585, 2.7935357e-05, 0.016836632, 0.017698102, -0.00024505914, -0.0053144977, 0.033470567, -0.00018013234, -0.013264074, -0.010660667, -0.029898008, -0.010039903, -0.019813765, -0.027186917, 0.04140114, -0.0061727986, -0.013441435, 0.035649575, -0.008792042, 0.013314749, 0.0049376055, 0.023297643, 0.001485399, -0.015367069, 0.0030024694, -0.0006559984, -0.009254447, -0.002445049, 0.01232026, -0.009533158, -0.026072077, 0.014720969, 0.012377269, 0.0066763773, -0.02483055, 0.001946221, 0.014847655, -0.0011607648, 0.052296177, 0.0062899836, 0.010172924, -0.016367894, -0.0011742252, 0.03795527, -0.014936335, 0.036587056, -0.036384355, -0.032609098, 0.013479441, -0.009178435, 0.0021631715, 0.010008232, 0.022600867, 0.008741367, 0.045379095, 0.0049946145, -0.037549872, 0.0029264577, 0.027060231, 0.0071324487, 0.016380562, -0.003011971, -0.018901622, 0.010850697, -0.017596751, -0.0009754859, -0.00782289, 0.035649575, 0.0062488103, 0.0128080025, -0.0018638748, -0.02698422, 0.006084118, -0.006261479, 0.033217195, -0.0105656525, -0.01095838, -0.0040983073, -0.0067777266, -0.029543286, 0.0120225465, -0.012415275, -0.007316144, -0.037423186, -0.013074044, -0.007107111, -0.025578, 0.01825552, 0.046139214, -0.0052131484, -0.01835687, -0.0006369954, 0.009051749, 0.0075631826, -0.0294166, 0.026756184, 0.01938303, 0.0065053506, -0.01442959, -0.011693162, 0.001050706, 0.019231007, -0.00843732, -0.024513833, 0.02436181, -0.0053335004, -0.005571038, 0.027389616, -0.023107613, -0.034129336, -0.014024193, -0.014188886, -0.044086892, 0.013264074, -0.0013373342, -0.006575028, -0.011085067, -0.01573446, -0.00936213, -0.05508328, 0.010052572, 0.008741367, 0.034990806, 0.034788106, -0.0046493937, 0.020586552, -2.6920876e-05, 0.012199908, -0.01545575, -0.008576674, 0.008722364, -0.028327096, 0.0011876858, 0.030962175, -0.026072077, -0.051561397, -0.004491036, 0.018597575, -0.0066573746, 0.0011457208, -0.0059827687, 0.0009984478, -0.00070825656, -0.009051749, 0.001564578, -0.0044276924, 0.016760621, -0.028732494, -0.007240132, -0.009621838, 0.013821495, -0.011522135, -0.032609098, 0.0013412931, -0.009400137, -0.021891423, 0.043225426, -0.012358266, 0.008513331, -0.023791721, 0.005419014, 0.02014315, 0.0018322031, 0.024045093, -0.04560713, 0.0005740481, 0.0010261604, 0.02903654, -0.027338943, 0.02885918, 0.01489833, 0.03869005, 0.01048964, 0.03146892, -0.0063058194, -0.009450811, 0.0001239152, -0.030886162, 0.052448202, -0.016684609, -0.045885842, -0.014834986, -0.007189458, -0.0046683964, 0.014112874, 0.02078925, -0.00355989, 0.011116738, -0.057110265, 0.01545575, 0.0032146694, 0.0019335523, 0.0132134, 0.017812118, 0.016557923, -0.0076772003, -0.035472214, 0.0009770695, -0.011142076, -0.026376124, 0.004779247, -0.01947171, 0.041071754, 0.04806485, 0.040919732, -0.0028409443, -0.0077532125, -0.022537524, 0.011655156, 0.0023246969, -0.010255271, -0.014100205, 0.0086906925, 0.015810473, 0.01157281, 0.017482733, -0.0153544005, -0.00059503055, 0.004614555, 0.042009234, 0.00025515447, -0.015075691, -0.0048172534, 0.0083296355, 0.010736679, -0.016912645, -0.019129658, 0.038436677, 0.003499714, -0.018040154, 0.056350145, -0.023170957, 0.030810151, 0.021815412, -0.008139607, 0.045049712, -0.03775257, -0.0067587234, 0.016811296, 0.00012916082, -0.024653189, -0.010394625, 0.020485204, -0.013441435, -0.01221891, -0.020776583, -0.0001080794, -0.0016152526, 0.02144802, 0.009970225, -0.0026002398, -0.019243676, 0.018052824, -0.0025147265, 0.015025016, 0.048571594, 0.031342234, -0.0033761945, 0.018888954, 0.029644636, 0.040412985, 0.045404434, 0.0012541962, -0.010755681, -0.033039834, 0.012073222, 0.013973518, -0.0037657556, -0.006109455, 0.017394053, 0.019446375, 0.0068980786, -0.004456197, 0.0436815, -0.029467275, -0.023335649, -0.008766704, 0.0054855244, 0.01695065, 0.041375805, -0.00590359, -0.0016485078, -0.0049977815, 0.011940201, 0.019877108, 0.0029913844, 0.01743206, -0.021346672, -0.013200731, -0.016697278, -0.004015961, 0.027313605, 0.0010063658, 0.019535055, -0.023373654, -0.04271868, -0.0138595, 0.008373976, 0.015316395, 0.007892568, 0.021384677, 0.009685181, -0.024589846, 0.018762266, -0.017900798, -0.014340909, 0.015392407, -0.018787604, 0.025071254, -0.016633935, 0.01432824, -0.0049249367, 0.009773862, -0.023779051, -0.033622593, 0.02220814, 0.002584404], "content_hash": "844343e7d8b8fdc70c74179aff4cb513"}, {"content": "like **\"The\"**, **\"of\"**, and **\"is\"** should be lighter -- they contribute less to this specific prediction. **Bar chart**: Shows the same information as horizontal bars with scores. Longer bars = more influence. **Hover over any token chip** to see the exact attribution score. ### Step 5: Compare with Integrated Gradients\n\n1. Now switch to **\"Integrated Gradients (more accurate, slower)\"**.\n2. Click **\"Compute Attribution\"** again.\n3. Compare the results. Integrated Gradients should give a more refined picture:\n   - The relative ordering of token importance may shift slightly\n   - Integrated Gradients tends to produce more reliable scores, especially for distinguishing tokens of moderate importance\n\n### Step 6: Change the Target Token\n\n1. In the **Target Token** dropdown, select a different token from the top-5 predictions (e.g., if the model also considered \"a\" or \"the\" as alternatives to \"Paris\").\n2. Run attribution again.\n3. **What to look for**: Different target tokens are driven by different input tokens. For example:\n   - \"Paris\" might be strongly driven by \"France\" and \"capital\"\n   - A generic token like \"the\" might be driven more by \"is\" (as a common grammatical continuation)\n\n### Step 7: Try a Different Prompt\n\nRun attribution on: `Alice gave the book to Bob because she`\n- Which tokens drive the prediction of the next word?\n- Does \"Alice\" have high attribution (suggesting the model connects \"she\" to \"Alice\")?\n- Does \"Bob\" have lower attribution than \"Alice\" for this prediction?\n\n## What You Should Learn\n\n- Token attribution reveals which input tokens \"caused\" a particular prediction\n- Content words (nouns, verbs) typically have higher attribution than function words (the, of, is)\n- Different target tokens can be driven by completely different input tokens\n- Integrated Gradients is more accurate but slower; Simple Gradient gives a quick approximation\n- Attribution helps you understand the \"why\" behind a model's prediction\n\n## What's Next?", "source_file": "experiment_token_attribution.md", "chunk_index": 1, "embedding": [0.006119887, 0.007317256, 0.0013586814, -0.010031293, 0.022723407, -0.018998258, 0.0062529277, 0.033260256, -0.0060500405, 0.0068250042, 0.014940506, 0.009033485, -0.041508798, -0.062901795, -0.025876477, -0.015752057, -0.0032112775, -0.0033642747, -0.02160586, 0.061518166, -0.05273746, -0.017268725, 0.043850318, 0.0369854, 0.027326625, -0.004157532, -0.019264339, -0.008554538, -0.0019074756, 0.02354826, 0.022204546, -0.016989337, -0.025064928, -0.05614331, -0.003781691, -0.0078095077, -0.026262296, 0.002998412, 0.012871718, 0.024599284, 0.00059452705, 0.003377579, -0.04470178, -0.011115577, -0.03421815, 0.017721064, 0.00733056, -0.04946465, -0.040098563, 0.096481346, -0.039193884, 0.03775704, -0.029109374, 0.018173402, -0.035309087, 0.020235538, -0.020248843, 0.0074968613, -0.023694605, -0.04251991, -0.031317856, -0.032222535, 0.037836865, 0.015911706, 0.007995765, -0.02416025, -0.047921374, 0.007343864, -0.0158984, -0.022257762, 0.027699139, 0.024798846, 0.04084359, -0.0019490509, -0.06040062, -0.03200967, 0.011148837, 0.034776922, 0.0016497086, -0.014075739, 0.020421796, -0.017375156, 0.009891599, -0.028790075, 0.00053964765, -0.047362603, -0.031184815, -0.04890588, -0.05231173, -0.06354039, -0.0150070265, -0.019716678, -0.020794312, 0.020461708, 0.014022524, -0.016696647, -0.022271067, -0.01933086, 0.02088744, -0.005078841, 0.016244309, -0.027752357, 0.008827272, -0.02902955, 0.023282178, -0.024439635, 0.018093579, -0.0074103847, -0.010370547, -0.00094542274, -0.031078381, 0.038768154, -0.06981993, 0.04635149, -0.025676915, -0.033872243, -0.044355877, 0.034244757, 0.023947382, -0.032994173, -0.029641539, -0.034138326, -0.0036785842, 0.040524293, 0.037544176, -0.009226395, 0.01018429, -0.048027806, -0.005750698, -0.026288904, -0.008468061, 0.0031963105, -0.0026425272, 0.000784942, -0.0019174537, -0.022138026, -0.0025194644, 0.035202652, -0.057526935, 0.039619613, -0.024439635, -0.056409393, -0.0034357843, 0.0024679108, -0.071469635, 0.009033485, -0.0017661195, 0.036852363, -0.05420091, 0.022630276, 0.030599434, -0.012020256, -0.036692712, -0.014567992, -0.04265295, -0.045792717, -0.0040012086, 0.029269025, -0.026036127, -0.05422752, 0.0044768304, 0.037224878, 0.009299567, -0.028949725, -0.024772238, 0.031211423, -0.044488918, -0.011840651, -0.027459666, -0.03775704, -0.04188131, -0.026541684, 0.013131148, -0.044994473, 0.023654694, -0.01829314, -0.05763337, -0.030918732, 0.04145558, -0.021313172, -0.030120486, 0.0171889, -0.0019141276, -0.012126689, -0.0111222295, -0.0073904283, -0.036639497, 0.02095396, -0.007669815, -0.0038615155, 0.044914648, 0.010729758, 0.075141564, 0.022204546, -0.05082167, -0.014980419, 0.051486872, 0.04339798, 0.025823262, 0.0049324958, 0.025357617, 0.03123803, 0.05954916, -0.031770196, -0.013284146, 0.017454982, 0.02030206, -0.01871887, -0.068649165, -0.00051802845, -0.0029834448, -0.06683981, 0.031903237, 0.020222235, -0.087753855, -0.012146645, -0.041136283, 0.025703525, 0.041854706, 0.033978675, 0.01329745, 0.033100605, 0.0191313, -0.025730133, -0.055291846, 0.028497387, 0.058857348, -0.02636873, 0.024865367, -0.049384825, 0.038422246, 0.04310529, -0.006525662, -0.048081025, -0.010343939, -0.05795267, 0.015219892, -0.032355577, 0.039007626, -0.055345062, -0.0027822203, 0.013084584, 0.029162591, -0.008873836, -0.011714262, 0.021685686, -0.021366388, -0.030998558, 0.023721213, 0.028683644, 0.047149736, -0.02948189, -0.033286862, 0.003397535, 0.019610247, -0.02286975, -0.0058271964, 0.026408643, 0.012372815, -0.03004066, 0.0026841026, 0.0034125021, -0.047256168, -0.011441528, -0.013144452, -0.017055858, -0.032408793, 0.019038169, -0.027645923, -0.043052074, -0.003395872, -0.0029019571, 0.01696273, 0.00018303534, -0.034005284, -0.0113217905, -0.044302657, 0.014395038, -0.025091536, -0.0151932845, -0.0050189723, 0.00499569, 0.06934098, 0.015326325, -0.039353535, 0.0042573125, 0.006006802, 0.03051961, -0.01576536, -0.0025926367, 0.01585849, -0.04991699, 0.026288904, -0.03227575, -0.023295483, 0.01232625, 0.026222384, -0.005604353, -0.03153072, -0.01333071, 0.011754174, -0.022084808, -0.0035422172, -0.017574718, 0.029348848, 0.0026441903, 0.030918732, 0.041056458, 0.02873686, -0.022138026, -0.012771938, 0.009159874, -7.566708e-05, 0.017175596, -0.02471902, 0.02393408, 0.06370004, -0.04621845, 0.022031592, 0.024093729, -0.02993423, -0.050316114, 0.014940506, 0.014461558, -0.0003666943, -0.017202204, -0.0010177637, 0.0074303406, -0.0010817897, -0.011607829, -0.0016838004, 0.03496318, 0.01076967, 0.06311466, -0.05954916, -0.022031592, -0.036240373, 0.0040178387, 0.009425956, 0.06071992, -0.050741844, -0.058431614, 0.02286975, -0.016177788, 0.025863174, 0.015552495, 0.0043837014, 0.028550602, 0.03509622, -0.007317256, 0.00019800245, 0.005630961, 0.018705567, -0.008162066, 0.006089953, 0.042280436, -0.011142185, -0.013324058, 0.010969232, 0.00019550794, -0.014182173, 0.031051774, 0.039885696, 0.02490528, -0.007343864, -0.013576835, -0.016137876, 0.015499279, 0.00053216406, -0.0064558154, -0.030599434, 0.0034756965, -0.041269325, -0.022390803, -0.05566436, 0.0014634512, -0.0019839741, 0.011148837, -0.02501171, 0.0140358275, 0.045420203, -0.046245057, 0.002629223, -0.027778964, 0.048799444, -0.011248618, 0.015086851, -0.03200967, -0.026342122, 0.03035996, -0.014607904, -0.01865235, -0.027592707, 0.03419154, -0.008800663, 0.03020031, 0.010716454, 0.053509098, 0.04706991, 0.06087957, -0.039087452, -0.010084509, -0.05747372, -0.01583188, 0.04220061, 0.041375756, 0.00017752661, 0.009465869, 0.0118073905, 0.03256844, 0.06327431, -0.035282478, 0.04102985, -0.030599434, 0.008241891, 0.028790075, 0.031477503, 0.008986921, 0.0408702, 0.025517266, 0.008095546, -0.060187757, -0.008162066, -0.07067139, -0.035309087, -0.00755673, 0.026262296, -0.017854104, 0.018439485, -0.047335993, 0.006542292, 0.035202652, 0.05364214, -0.039805874, 0.0047395863, -0.02118013, 0.062742144, 0.036373414, 0.01858583, -0.018918432, 0.031903237, -0.05135383, -0.031637155, -0.056196526, -0.055185415, -0.021885248, -0.033872243, -0.05614331, -0.008880488, 0.0062429495, -0.033978675, 0.015512583, -0.0078095077, 0.05566436, -0.014155564, -0.04121611, 0.03094534, 0.016976034, 0.059016995, -0.023694605, 0.034031894, 0.0059635635, -0.01680308, 0.01933086, 0.007856072, 0.0074103847, 0.0027057217, -0.02393408, 0.03775704, -0.004303877, 0.013031367, -0.010483632, -0.006166451, -0.05185939, 0.021738902, -0.026501771, 0.018466093, -0.015259805, 0.008328368, -0.025477355, -0.016656736, 0.008461408, -0.012951543, 0.011780782, -0.018998258, 0.064977236, 0.03315382, -0.014208781, -0.0022716753, -0.031184815, -0.00054047914, -0.057260852, 0.020089194, -0.014820769, 0.017375156, -0.023694605, 0.014128956, -0.018133491, 0.022656886, -0.0126788085, 0.042307045, -0.03123803, -0.052205294, 0.07115033, -0.016390653, -0.0025760068, 0.050156463, 0.029109374, -0.03182341, 0.047575466, -0.047442425, -0.03969944, -0.031903237, 0.00943926, -0.025091536, 0.00651901, -0.018106882, 0.055717576, 0.040364645, 0.015525887, -1.6279335e-05, 0.027433058, -0.008720838, -0.0060334103, 0.0013753115, -0.01770776, -0.02451946, -0.015978226, -0.03216932, -0.042839207, 0.022164633, -0.026927503, 0.0054247477, 0.0048859315, -0.04664418, 0.002486204, 0.003788343, -0.030173702, -0.044409093, 0.014767553, 0.016856296, 0.0040377947, 0.0050588846, 0.03717166, -0.005657569, 0.0072440836, -0.008115502, -0.034697097, -0.0042273784, 0.010510241, -0.009026833, -0.026009519, 0.022537148, -0.03065265, 0.0005687504, -0.03613394, 0.022470627, -0.011767478, -0.022563757, -0.028710252, 0.024173552, 0.006013454, 0.013430491, -0.0066021606, -0.014514775, 0.045366988, 0.042253826, 0.007184215, 0.023069313, 0.0046963478, 0.010955928, 0.010869451, 0.031025166, 0.011867259, 0.032754697, 0.01748159, -0.009851688, 0.004849345, 0.008428148, -0.004397006, -0.013450447, -0.009652126, -0.009272959, 0.0097785145, -0.02474563, -0.015313021, 0.01651039, 0.008215283, 0.0039712745, 0.02844417, 0.0062462757, -0.021060392, 0.004177488, 0.013111192, -0.025038319, 0.0265949, 0.0017212181, -0.019157907, -0.020275451, 0.014288605, 0.039167274, -0.032382183, -0.031291246, -0.0015299717, -0.002210144, 0.021392995, -0.00862771, 0.014900594, -0.011261922, 0.0048626494, -0.013390578, -0.054387167, -0.020035977, -0.00013293709, -0.008441453, -0.004180814, -0.031025166, 0.005996824, 0.016696647, -0.039593007, 0.0064425115, -0.009971424, -0.006013454, -0.0009778515, -0.010869451, 0.040364645, 0.0029019571, -0.00032491112, 0.022177938, -0.0008061454, -0.027246801, 0.0010385514, 0.019942848, 0.007689771, 0.01284511, 0.012053516, -0.019543726, 0.021911856, 0.0070179137, 0.0007105222, -0.0040078606, 0.02377443, -0.015778664, -0.023268875, -0.011674349, -0.0055511366, 0.010210898, 0.023388611, -0.015432758, -0.0291892, -0.03419154, -0.033366688, 0.048240673, -0.016497087, -0.0014476526, -0.023441827, 0.027246801, 0.016377348, -0.03395207, -0.013729833, -0.027353233, -0.015459366, -0.03820938, -0.042040963, -0.015366238, 0.013822962, 0.0071309987, 0.03259505, 0.022417411, -0.008315064, -0.10244159, 0.024918582, 0.0002673293, 0.0363202, -0.0071509546, -0.011082317, 0.008441453, -0.054280736, -0.02993423, 0.03781026, -0.012233121, -0.0064724456, -0.040896807, -0.033419903, 0.00033551283, -0.021951769, 0.02636873, 0.00061905646, 0.006286188, -0.030971948, -0.020488316, -0.029907621, 0.0069713495, -0.007856072, -0.024200162, -0.00733056, 0.020634662, 0.053934827, -0.016616823, -0.08754099, -0.004214074, -0.003794995, -0.027619315, -0.02156595, -0.0044036577, 0.0026574943, -0.013290797, 0.03509622, 0.0090933535, -0.03392546, -0.034564056, -0.0031214748, 0.016111268, -0.010077857, -0.021246651, 0.017228812, 0.0048925835, 0.017534805, -0.010696498, -0.060613487, -0.026009519, -0.00071800576, -0.04015178, 0.0015133416, 0.024253378, -0.013377274, -0.020541534, 5.3320346e-05, -0.02137969, 0.019729983, -0.007224127, 0.003415828, -0.012419379, -0.045074295, -0.006415903, 0.010490284, -0.026648115, 0.03363277, -0.025291098, -0.016856296, 0.0045533287, 0.0013171061, 0.0022550453, -0.017135683, 0.025663612, 0.03701201, -0.036825754, 0.028710252, -0.022936272, -0.041375756, -0.019317556, 0.011647741, 0.0150070265, -0.011528004, 0.015605711, 0.0037683868, 0.0047429125, -0.04488804, -0.00862771, 0.014408343, 0.004836041, 0.03123803, -0.012073472, 0.0039047538, -0.01998276, -0.018625742, 0.027672531, 0.026541684, -0.031876627, 0.058591265, -0.0012073473, -0.008601102, 0.03866172, -0.028364345, 0.03451084, -0.003645324, 0.022244459, 0.011840651, -0.019024866, 0.0040976633, 0.0029119353, -0.0010610021, -0.02286975, 0.009831731, 0.025996216, 0.0074968613, -0.0008930378, -0.036932185, -0.008747447, -0.00755673, -0.020807615, 0.0035621733, 0.011182098, -0.02251054, -0.0043338113, 0.025996216, -0.010836191, -0.1151603, -0.012924935, 0.059123427, -0.020754399, -0.001566558, 0.0008780707, 0.046750613, 0.005890391, -0.036187157, 0.023308786, -0.02695411, -0.019889632, -0.022590365, -0.016670039, -0.011853955, -0.008813967, -0.012266382, -0.01511346, -0.013623401, -0.014687728, -0.017002642, 0.0036386717, 0.00817537, -0.03203628, -0.051619913, -0.0071243467, -0.02740645, -0.01897165, 0.013064628, 0.01329745, -0.010383852, 0.007689771, 0.02416025, 0.0005055559, 0.014408343, 0.021233346, -0.0022517191, 0.016976034, 0.0033609488, -0.02977458, 0.01608466, -0.02857721, 0.038608503, -0.033659376, 0.0765252, -0.05954916, -0.003139768, -0.012592332, -0.008155414, 0.015752057, -0.025823262, 0.021007176, -0.0012971499, 0.008095546, 0.024865367, 0.019876327, 0.031876627, -0.0360009, -0.021326475, 0.022949576, -0.008102198, 0.013876178, -0.008441453, 0.00023282178, 0.0053249667, 0.010530196, -0.00546466, 0.01089606, -0.0428126, -0.015419454, -0.005870435, 0.0246525, -0.0037650608, -0.019357469, -0.024705717, 0.014581296, 0.0038781457, 0.003026683, -0.04459535, 0.02007589, 0.059336293, 0.00084314746, 0.0025560504, 0.040550902, 0.0048626494, 0.035921074, 0.0027339929, -0.011315138, -0.002107037, 0.034564056, 0.057793017, -0.008667623, 0.00836828, -0.0056642215, -0.0431319, -0.008155414, 0.028204696, 0.043318156, 0.025038319, -0.012798546, 0.008813967, 0.03983248, -0.04595237, -0.009612214, -0.014408343, 0.0010618336, -0.0057773064, -0.020634662, -0.007869376, 0.0015673895, 0.018772088, 0.03480353, -0.02030206, 0.008354976, -0.017441677, 0.02977458, 0.0070977383, 0.023561563, 0.023441827, -0.029348848, 0.017827496, 0.012924935, -0.008022374, -0.006685311, -0.017215507, 0.027326625, -0.018732175, -4.516119e-05, 0.008488017, -0.0014967114, 0.03123803, 0.002620908, -0.009319523, -0.015219892, -0.039140668, 0.00276559, 0.015738752, -0.024572676, -0.05672869, 0.027100455, -0.011195402, 0.02296288, 0.024559371, 0.006392621, -0.043451197, -0.020741094, -0.0431319, -0.04874623, 0.025317706, -0.00093877065, 0.032754697, 0.020315364, 0.000783279, 0.011248618, -0.024213465, 0.028311128, 0.025956303, -0.005930303, 0.017268725, 0.050316114, -1.0218434e-05, 0.03272809, 0.011315138, -0.013543576, 0.055930443, -0.038635112, -0.0018143469, 0.0025244532, -0.007450297, -0.011887215, -0.027193584, 0.0043936796, 0.013337362, 0.039885696, 0.022044897, 0.030838909, 0.009040137, 0.0046597617, 0.0072773434, 0.02715367, -0.0022949576, 0.013969307, -0.0071443026, 0.018027058, 0.03347312, -0.027326625, 0.0027838834, 0.0052551203, -0.0015939977, 0.000117242394, 0.009166526, 0.0230294, 0.012838458, 0.050449155, -0.03674593, 0.0018659002, 0.031158207, 0.010576761, 0.009299567, 0.0136500085, -0.045233946, -0.0031031817, 0.035149436, 0.0041375756, 0.022204546, 0.01770776, -0.02682107, -0.0008780707, 0.0084348, -0.0007932571, -0.015259805, 0.0096787345, -0.032062884, 0.042040963, -0.0068782205, -0.001834303, -0.032834522, -0.00021234594, -0.0017627935, -0.02007589, -0.05332284, -0.02828452, -0.005780632, 0.0036952142, -0.0018742153, -0.00274896, 0.056835122, -0.01284511, 0.008388236, -0.0017677825, -0.014248693, -0.007929245, -0.013516967, -0.015619015, -0.01917121, 0.019902937, -0.007463601, -0.00042614702, 0.018612439, 0.0066753333, 0.039619613, 0.026248993, -0.009425956, -0.01066989, -0.0048759533, -0.025118144, 0.017242115, 0.023495045, -0.0051819477, 0.003505631, 0.0123927705, 0.059815243, 0.048081025, -0.037411135, -0.0011375007, -0.018692262, 0.030466393, 0.026022824, 0.014554687, 0.020860832, 0.022457324, 0.025344314, 0.010323983, -0.02449285, 0.0042706165, -0.02127326, -0.007716379, 0.010011337, 0.01998276, 0.010117769, -0.026036127, 0.03586786, 0.0022982836, 0.036666103, 0.007303952, -0.0071243467, 0.005667547, -0.03879476, 0.008873836, -0.004609871, -0.014993723, 0.01533963, -0.041109674, 7.254893e-05, 0.009272959, 0.0035023047, 0.00057249214, -0.01284511, -0.017960537, -0.0031198119, 0.0037018664, 0.011833998, 0.012173253, 0.0046298276, 0.0059569115, 0.0074236887, 0.017614631, 0.005334945, 0.01261894, -0.028790075, 0.027353233, 0.0026591574, -0.0070378697, -0.012366163, 0.009618865, 0.035335694, 0.016483782, -0.027353233, 0.008255195, -0.003918058, -0.039167274, -0.00050389284, -0.020328667, 0.01576536, 0.0059569115, -0.023721213, -0.017268725, -0.03983248, -0.02257706, -0.016403958, 0.0028088284, -0.0067252233, -0.0017378483, 0.035601776, 0.006419229, -0.004955778, -0.0011042404, 0.0148074655, -0.015073547, 0.005477964, -0.025264489, -0.008155414, 0.008900444, -0.0041309237, -0.050422546, 0.014860681, -0.028417561, -0.01709577, -0.008022374, -0.0431319, 0.012924935, 0.020009369, 0.0094924765, -0.032089494, 0.034271367, 0.008940357, 0.0071576065, 0.01044372, 0.01858583, 0.017335245, 0.080942154, -0.018825304, 0.005318315, 0.010403807, -0.01378305, 0.025703525, -0.012352859, 0.009292915, -0.04121611, 0.013397231, -0.0030682585, 0.02474563, 0.036187157, -0.0066986154, 0.00016432644, 0.0062595797, 0.011873911, 0.009745254, 0.029348848, 0.021033784, 0.0056941556, 0.04028482, -0.027233496, 0.012725374, 0.005734068, 0.023428524, 0.012046864, 0.008680927, 0.016590215, 0.013956003, 0.011581221, -0.020341972, -0.01284511, 0.016909514, -0.03349973, 0.016430566, -0.02630221, 0.026515074, -0.024080424, 0.0059868456, 0.011614481, 0.021353083, -0.0060500405, 0.0063360785, -0.009825079, 0.016829688, 0.030306743, -0.02620908, -0.007962505, -0.019956153, -0.006655377, 0.0052784025, -0.023521652, -0.019117994, 0.005627635, -0.0012406075, -0.005341597, -0.013576835, 0.0063959467, -0.020395188, 0.004140902, 0.016550303, -0.019769896, -0.0017627935, 0.020501621, -0.041668445, 0.0043837014, -0.013390578, 0.0015657265, 0.019543726, -0.035761427, -0.013729833, -0.012179906, -0.0012830143, 0.015592407, 0.0071043903, -0.0038548636, -0.0051786215, 0.024373114, -0.008986921, -0.013131148, -0.021419603, -0.04031143, -0.00039808368, -0.028923117, -0.016616823, 0.05050237, -0.0012189883, -0.021845335, 0.030812299, -0.027965222, 0.001239776, 0.027672531, 0.012199861, 0.008973616, -0.024891974, -0.015565799, -0.0014094033, -0.01333071, -0.0037983211, 0.017082466, -0.01229299, -0.029428674, -0.011860606, 0.008341672, 0.0023199026, -0.05183278, -0.007895985, 0.017268725, -0.0019307578, 0.04017839, -0.010403807, 0.023175744, -0.0024512808, -0.0091066575, 0.0090933535, 0.0027256778, 0.051619913, -0.03363277, -0.041934527, 0.0030699214, 0.0060633444, -0.0005126237, -0.01517998, 0.0065090316, 0.013995915, 0.014567992, 0.012539116, -0.03703862, -0.001172424, 0.016616823, 0.021007176, 0.020501621, 0.005225186, -0.004872627, 0.002474563, 0.0078028557, 0.0043737236, -0.008867184, 0.014634512, 0.010543501, 0.011501396, -0.014448254, -0.036346804, 0.019211123, -0.011973691, 0.0291892, 0.002614256, 0.004693022, 0.007822812, 0.0007013756, -0.021685686, 0.014900594, -0.012459291, 0.007197519, -0.025384227, -0.016324133, -0.011641089, 0.00020933173, 0.014514775, 0.03395207, -0.01261894, -0.009139918, -0.010057901, 0.017175596, -0.0014202129, -0.03509622, 0.026967414, 0.020155715, -0.013862874, -0.011780782, -0.015778664, -0.022550453, 0.026914198, -0.010310679, -0.032967564, 0.007583338, 0.031637155, -0.005075515, 0.031610545, -0.014501471, -0.029694755, 0.008048981, -0.010317331, -0.03453745, 0.015273109, 0.010463676, -0.009745254, 0.0034823485, -0.008115502, -0.0071043903, -0.07189537, -0.0079492, 0.0046497835, 0.029614931, 0.034324583, -0.019197818, 0.014940506, 0.0035921074, -0.0018659002, -0.01585849, 0.006129865, 0.036666103, -0.022244459, -0.00033655218, 0.037730433, -0.013430491, -0.036719322, 0.0032395488, 0.027087152, 0.0046497835, 0.0043870276, 0.009692038, 0.023801038, 0.028550602, -0.011241966, -0.00651901, -0.016310828, 0.01579197, -0.026807765, -0.0010327309, -0.016816385, 0.016670039, -0.0020837549, -0.030120486, -0.0041741617, 0.0023199026, -0.018891824, 0.033659376, 0.0067218975, 0.023388611, -0.035309087, -0.022364195, 0.025211273, 0.026049431, 0.014780857, -0.043903537, -0.01897165, 0.0023947384, 0.039779264, -0.016137876, 0.015619015, -0.016151179, 0.016377348, 0.004533373, 0.015060243, 0.004842693, -0.008095546, 0.004067729, -0.010111118, 0.035734817, -0.0062429495, -0.04076377, -0.021126913, -0.0118073905, -0.028896509, 0.020501621, 0.022829838, 0.0012289664, 0.012386119, -0.06449828, 0.008840576, 0.012605636, -0.0050522326, 0.013876178, 0.03126464, -0.003026683, -0.0030599434, -0.024146944, -0.0039014278, -0.009166526, 0.0013279157, -0.010643281, -0.030280136, 0.032515224, 0.042147394, 0.016403958, -0.019224428, 0.0041109673, -0.023122529, 1.7201788e-05, -0.00061115716, -0.005231838, -0.03035996, 0.01488729, 0.01336397, 0.0029152613, 0.020421796, -0.021725599, 0.0026275602, 0.012771938, 0.030546218, 0.0046364795, -0.026634812, -0.017734367, 0.00840154, 0.015219892, -0.02393408, -0.0062296456, 0.014554687, 0.012851763, -0.004829389, 0.035229262, -0.018013753, 0.023987295, 0.03554856, -0.0021851987, 0.04039125, -0.03153072, -0.009638822, 0.02902955, -0.0012580691, -0.020089194, -0.001962355, -0.0028005135, -0.0068715685, -0.015738752, -0.011647741, -0.00036108165, -0.0059036952, -0.005630961, -0.005637613, 0.019450597, -0.0017228812, 0.0064491634, 0.009266307, 0.0088073155, 0.049943596, 0.005887065, 0.0039879046, 0.010270767, 0.0389278, 0.037730433, 0.013902786, 0.013510316, -0.005630961, -0.023016095, -0.0026890915, 0.0126788085, -0.011900519, -0.009519084, 0.011880563, 0.005388161, -0.013091236, 0.0035654993, 0.039459966, -0.027858788, -0.015446062, -0.0291892, 0.0073638204, 0.014953811, 0.03331347, 0.011501396, -0.015073547, 0.0030848887, 0.017947232, 0.016497087, -0.0013062964, 0.0075500775, -0.043770496, 0.0015449388, -0.022750014, -0.011661045, 0.016071355, -0.004895909, 0.014860681, -0.021712294, -0.050395936, -0.008527929, 0.011261922, 0.007283996, 0.0063261003, 0.0340585, -0.0014326854, -0.015406149, 0.017814193, -0.013257537, -0.014940506, 0.013430491, -0.009186482, 0.03166376, -0.008135458, -0.008441453, -0.008328368, 0.01770776, -0.031477503, -0.025863174, 0.014913898, 0.005584397], "content_hash": "9eb7f48406d75b5edf6310b1bcf2c8b2"}, {"content": "typically have higher attribution than function words (the, of, is) - Different target tokens can be driven by completely different input tokens - Integrated Gradients is more accurate but slower; Simple Gradient gives a quick approximation - Attribution helps you understand the \"why\" behind a model's prediction ## What's Next?\n\nMove on to **Experiment: Comparing Heads** to combine ablation with your understanding of head categories, or try **Experiment: Beam Search** to explore how the model generates longer sequences.", "source_file": "experiment_token_attribution.md", "chunk_index": 2, "embedding": [-0.01083987, 0.036451172, 0.041786857, -0.0038999664, 0.019419095, -0.009070669, 0.007975449, 0.011914029, -0.0021307648, 0.022803044, 0.02375785, -0.0024326523, -0.034457307, -0.027113717, -0.041983437, -0.02069685, 0.014322109, -0.049060244, 0.007182116, 0.05287947, -0.033025097, -0.030413419, 0.028251061, 0.064983055, 0.03369908, -0.014476563, -0.033446338, 0.015389246, 0.012742464, -0.00186047, 0.018955732, -0.025667466, -0.032014124, -0.04858284, 0.0023396288, 0.008565183, 0.014869719, 0.017411191, 0.013325178, 0.0011680592, 0.017144406, -0.005314626, -0.079698324, -0.012272081, -0.02024753, -0.006311557, -0.013767478, -0.038782023, -0.01971396, 0.08261891, -0.035636775, 0.047740363, -0.03195796, 0.053441122, -0.026846934, 0.0310874, -0.009295329, -0.02219927, -0.00689778, -0.014230841, -0.031620968, -0.027787698, 0.040888216, 0.027408585, -0.0017753447, -0.03128398, -0.051391095, 0.021904401, -0.022999622, -0.013591962, 0.008354563, -0.0017990394, 0.047543786, -0.015276916, -0.047431454, -0.026776727, 0.017355027, -0.0054234457, 0.0011136492, -0.020921512, 0.0136551475, -0.0072523225, -0.004212385, -0.03426073, -0.02026157, -0.043584142, -0.017003994, -0.05908572, -0.055434987, -0.047768444, -0.0011584059, 0.025246227, -0.024530122, 0.0061746542, -0.001187366, 0.018731073, -0.0136551475, -0.019826291, 0.017116325, 0.0051566614, -0.027928112, -0.011661286, 0.019208476, -0.02878463, 0.04378072, -0.056530207, -0.028054483, -0.030413419, -0.017607769, 0.012833733, -0.009330433, -0.008796864, -0.04355606, 0.05417127, 0.010011435, -0.038669694, -0.018085172, 0.04847051, 0.017635852, -0.027661327, -0.0014199248, -0.018464288, 0.0031698197, 0.004117606, 0.018422164, -0.023140034, 0.04170261, -0.051475342, -0.00448619, 0.0017902636, -0.0052619707, 0.020079035, -0.0024975932, -0.010720519, -0.029711355, 0.008881112, -0.012117627, 0.038978603, -0.077901036, 0.055631563, -0.010130785, -0.052008912, 0.0059605245, 0.006062324, -0.06779131, 0.0040649516, -0.010769664, 0.039540254, -0.047684196, 0.027871948, -0.012777568, -0.017888594, -0.039203264, -0.012932022, -0.053469207, -0.05144726, -0.031115483, 0.035608694, 0.00831946, -0.034962796, 0.022620507, 0.018127296, 0.03852928, -0.022592425, -0.030862741, 0.029599024, -0.077620216, 0.000859151, -0.016400218, -0.01981225, -0.036928575, -0.026495902, -0.038445033, -0.019573549, 0.025330475, -0.012770547, -0.052598648, -0.017509481, -0.019868415, -0.032014124, -0.0064625004, 0.034822382, 0.008691554, 0.009119813, -0.004531824, -0.044510867, -0.01565603, 0.015417329, -0.0030101, 0.010376508, 0.014532728, 0.045269098, 0.091493, 0.0013181254, -0.0061535924, -0.030357255, 0.057513095, 0.040747803, -0.0009039076, 0.008052676, 0.032435365, -0.007568252, 0.063635096, -0.028208937, -0.032210704, 0.033783328, 0.030441502, 0.021904401, -0.05349729, -0.02165166, 0.01903998, -0.056867197, 0.01499609, 0.017888594, -0.085427165, -0.031592887, -0.01816942, 0.021202337, 0.042376593, 0.017481398, 0.0106994575, 0.009913146, 0.031340145, -0.008340522, -0.066106364, 0.019840334, 0.05987203, -0.056979526, 0.033193592, -0.047234878, 0.039764915, 0.018478328, -0.010629252, -0.046251986, -0.028756548, -0.061163828, -0.011218986, -0.03381141, 0.05756926, -0.047740363, -0.034822382, -0.007287426, 0.0066064238, -0.013388364, -0.018029008, 0.026229117, -0.026369529, -0.02549897, 0.036928575, 0.009807836, 0.030666163, -0.009337453, -0.04981847, 0.025133897, 0.0033751735, -0.012194854, 0.0038262496, 0.013739396, 0.017158449, 0.00015324744, 0.014083407, 0.011935091, -0.014434439, 0.025990415, -0.032182623, -0.023294488, -0.011478749, 0.012356329, -0.03569294, -0.035412118, -0.022255434, -0.03400799, 0.028517846, -0.0067257746, -0.015052255, -0.002309791, -0.010889015, 0.033951823, -0.023280447, -0.028068526, -0.00843179, 0.005588431, 0.06307344, -0.016105352, -0.057204187, 0.008895153, -0.016681043, 0.015810484, 0.006258902, -0.008417749, 0.0071610543, -0.04015807, 0.018534495, -0.023687644, 0.01432913, 0.038641613, 0.010713499, 0.004314184, 0.003053979, -0.012328247, 0.02715584, -0.016077269, 0.020949595, -0.002144806, 0.022522219, 0.012187834, 0.00043176947, 0.05897339, 0.02943053, 0.00037845646, 0.012201875, 0.018014966, 0.02023349, 0.028602093, -0.023589356, 0.008066718, 0.035945684, -0.03793955, 0.020064993, 0.017509481, -0.02121638, -0.075092785, 0.018365998, -0.012489721, 0.028180854, 0.0024975932, -0.036731996, 0.02167974, -0.011562997, 0.0028942595, -0.010411612, 0.004605541, -0.021061925, 0.03282852, -0.014925884, 0.01324093, -0.071835205, 0.043640308, 0.007203178, 0.045044437, -0.07234069, -0.029346282, -0.007792912, -0.0048407326, 0.031705216, 0.05144726, 0.031817548, 0.05262673, 0.03939984, -0.008516038, -0.0030609998, -0.019405054, 0.0065151555, 0.018969774, 0.016554672, 0.02287325, -0.0051917643, -0.03324976, 0.01488376, 0.009239164, -0.015347122, 0.022311598, 0.029570943, 0.04060739, -0.021595493, -0.023617439, 0.011675327, 0.01269332, -0.0086564515, -0.007820995, -0.024052719, -0.0049776346, -0.040775888, -0.0031470025, -0.043836884, 0.023673603, -0.009021524, 1.0071604e-05, -0.035861436, 0.0149118425, 0.008066718, -0.0105520245, -0.045212932, -0.058467902, 0.030385336, -0.026243158, 0.0031750852, -0.005735864, -0.03875394, 0.038894355, 0.008937277, -0.035805274, -0.031649053, 0.03063808, -0.015459453, 0.028588053, 0.032744274, 0.056726784, 0.04133754, 0.05484525, -0.032210704, -0.00045765808, -0.040944383, -0.04102863, 0.040747803, 0.050492454, 0.024263337, 0.050801363, 0.028152773, 0.020472191, 0.053834278, 0.00908471, 0.042208098, 0.0032909256, -0.0027240089, 0.0038122083, 0.007561231, -0.010334385, 0.02506369, 0.05341304, 0.0025993926, -0.04642048, -0.027197964, -0.052430153, -0.023069829, 0.017692016, 0.023477025, -0.017355027, 0.015164586, -0.018506411, -0.008031614, 0.02976752, 0.021792071, 0.007406777, -0.005570879, -0.011134737, 0.06341043, 0.060546014, -0.00448619, -0.023715727, 0.04465128, -0.024530122, -0.0396245, -0.049397234, -0.06442141, 0.012321226, -0.06093917, -0.045943078, -0.022480095, -0.014715265, -0.009393618, 0.014687182, -0.021398915, 0.022564342, -0.0009267247, -0.034288812, 0.040663555, 0.00492849, 0.07599142, -0.004222916, 0.023420861, -0.009913146, -0.005525245, -0.011780636, 0.028644217, 0.04080397, 0.007080317, -0.052233573, 0.040551227, -0.009878043, 0.023041746, 0.02878463, 0.021469122, -0.055828143, 0.0017753447, -3.7955342e-05, 0.028335309, -0.02976752, -0.010222054, -0.03515937, -0.030834658, 0.019011898, 0.02419313, 0.018674906, -0.022746878, 0.07419414, 0.0085581625, -0.021370834, 0.00246249, -0.009470846, -0.0115981, -0.060433682, 0.022058856, -0.016877621, 0.016610838, -0.022044815, -0.008586245, -0.012279102, 0.0069434145, -0.0026257199, 0.053104132, -0.02287325, -0.05714802, 0.060096692, 0.006774919, 0.0055603483, 0.044033464, 0.0161896, -0.023491066, 0.03942792, -0.03763064, -0.016400218, -0.018941691, 0.023799974, -0.010292261, 0.005111027, -0.026186993, 0.054789085, 0.03990533, 0.0161896, -0.011254088, 0.016596796, -0.026439736, -0.028419558, 0.0043808804, -0.020921512, -0.017060159, -0.047628034, -0.049930803, -0.016582755, 0.007498045, -0.04246084, 0.024165047, -0.016582755, -0.064983055, -0.007157544, 0.0072242403, 0.0049811453, -0.04465128, 0.015290957, 0.016835498, 0.0047705257, 0.03105932, 0.032238785, -0.010692437, 0.008600286, -0.0013856991, -0.02035986, 0.0122510195, 0.013921932, -0.00093286776, -0.044679362, 0.017312903, -0.044286206, -0.009253206, 0.0033277841, 0.012707361, -0.0042194054, -0.012651196, -0.025190063, 0.011864884, -0.0047670156, -0.007757809, -0.021749947, -0.02702947, 0.043752637, 0.02406676, -0.008551141, 0.040073823, -0.020710893, -0.0155437, -0.0024712658, 0.03872586, -0.029233951, -0.00145064, 0.033558667, -0.0043738596, 0.017481398, 0.016147476, -0.03240728, 0.0023326082, -0.030862741, -0.0115138525, -0.0023993042, -0.003020631, -0.03763064, -0.0005932442, 0.022831127, -0.0027204987, 0.02811065, 0.02461437, -0.015698155, 0.014616976, -0.007694623, -0.0151084205, 0.002888994, 0.007842056, 0.0055322656, -0.024965402, -0.0050373105, 0.04616774, -0.027183924, -0.011022408, -0.011296213, -0.011787658, 0.007982469, -0.05768159, 0.00084028306, 0.010102703, 0.005507693, -0.010678396, -0.015894733, -0.021637617, -0.0052268677, -0.028868878, -0.0033663977, -0.023392778, -0.014364232, 0.035608694, -0.049060244, -0.006413356, 0.00027358564, 0.018183462, 0.013142641, -0.019826291, 0.033979904, -0.01969992, -0.02812469, 0.0133111365, -0.009337453, -0.02670652, 0.024558203, 0.024221214, 0.011155799, 0.021426998, 0.0007196158, -0.030357255, 0.013170724, 0.01871703, 0.013065414, -0.029964099, 0.023477025, -0.011015386, -0.025091773, -0.01913827, 0.007055745, 0.0027169883, 0.005918401, -0.0032909256, -0.01882936, -0.026495902, -0.035018962, 0.043134823, 0.010348425, -0.011534914, -0.012700341, 0.016105352, 0.014574852, -0.017242696, -0.032744274, -0.030946989, -0.027324338, -0.040579308, -0.0475157, -0.029514777, 0.018450245, 0.006150082, 0.030609997, -0.0077086645, -0.02725413, -0.08458469, 0.032856602, -0.0028907491, 0.0323792, 0.0065397276, -0.023687644, 0.003533138, -0.04650473, -0.016175559, 0.050211627, 0.01378152, -0.0038578426, -0.03777105, -0.047094464, -0.010643292, 0.0036717956, 0.0042650397, -0.0026853953, 0.012321226, -0.019854374, -0.005483121, -0.014251903, -1.0750357e-05, -0.000221589, -0.007210199, -0.007926304, 0.01247568, 0.05768159, -0.0052022953, -0.016456384, 0.002255381, 0.00013350189, -0.012791609, -0.030048346, -0.023294488, -0.008838988, 0.0027626224, 0.06762282, 0.019615673, -0.038922437, -0.030918906, 0.0024923277, 0.010776685, -0.0009653382, -0.012377391, 0.030666163, 0.005044331, -0.0011724471, -0.01017291, -0.044510867, -0.025948292, -0.001641075, -0.025428765, 0.008628368, 0.013233909, 0.018239627, -0.013669189, -0.0023887733, -0.031817548, 0.009154917, 0.014743348, 0.006760878, -0.011176862, -0.030441502, -0.0017735895, 0.023041746, -0.034457307, 0.015220751, -0.017972842, -0.039203264, 0.03406415, -0.01313562, 0.007364653, -0.01203338, 0.0021886851, 0.015164586, 0.02121638, 0.017565645, 0.0033400701, -0.055266492, -0.013332198, 0.013261992, 0.0068767183, -0.003547179, 0.025948292, 0.0013979853, 0.026453778, -0.04552184, -0.018099215, -0.0017990394, 0.008487956, 0.017790306, -0.0020482722, 0.0010855667, -0.0029521799, -0.037237484, 0.011696389, 0.01565603, -0.022803044, 0.057513095, -0.0063326187, -0.021300627, 0.04321907, -0.034176484, 0.028166814, 0.0022378296, 0.038051877, -0.01302329, -0.027450709, 0.013360281, -0.00048968976, 0.0037139193, -0.024361625, 0.006378253, 0.0017077711, 0.0057604364, -0.003162799, -0.02277496, 0.0037139193, -0.06020902, -0.006869698, -0.0006432663, 0.016105352, -0.01641426, -0.002945159, 0.025625343, -0.010495859, -0.120867364, 0.0016779334, 0.021721864, -0.03347442, -0.015922815, -0.00033326106, 0.06807214, -0.0013137376, -0.006006159, 0.029711355, -0.0004218967, -0.006301026, -0.0032698638, -0.023154076, 0.026453778, -0.038613528, -0.023701686, -0.022311598, -0.024768824, -0.01630193, 0.0036928575, -0.014841637, 0.005430466, -0.03653542, -0.058467902, -0.0057639466, -0.028503805, -0.022957498, 0.01913827, 0.02659419, -0.017888594, -0.025920209, 0.013044352, 0.0028100119, 0.0002795093, 0.033895656, 0.0006476542, 0.0067257746, 0.024263337, -0.012728423, 0.012356329, -0.054704838, 0.033895656, -0.0023115461, 0.052739058, -0.04279783, 0.030160677, -0.0012163261, 0.004528314, 0.009555093, -0.0056445957, -0.010811788, -0.002429142, 0.019938622, 0.022058856, -0.008480935, 0.009330433, -0.012721403, -0.017958801, 0.045999244, 0.008024593, 0.014434439, -0.004289612, -0.012925001, -0.005876277, -0.02419313, 0.007891201, 0.038950518, -0.022817085, -0.04246084, 0.00021160653, 0.023224283, -0.018000925, -0.0069469246, 0.0075261276, -0.0061535924, 0.021735907, 0.008249254, -0.033305924, 0.03777105, 0.047094464, 0.0054480177, -0.03347442, 0.02857401, 0.001827122, 0.04872325, -0.020444108, -0.014673141, -0.026313365, 0.0378553, 0.04967806, -0.046897884, -0.0028854837, 0.022423929, -0.03743406, -0.023392778, 0.002464245, 0.016498508, 0.03743406, -0.00941468, 0.0022150124, 0.02604658, -0.029936016, -0.023041746, -0.018071132, -0.019292723, -0.013711313, -0.011773616, -0.009604238, -0.0034629314, 0.023715727, 0.031649053, -0.028714424, 0.0120544415, -0.016512549, 0.021497205, 0.0076876027, 0.0030030794, -0.0025835962, -0.0120544415, -0.018688947, 0.012616093, 0.005374301, 0.014673141, -0.008193089, 0.013465591, -0.0067222645, 0.0019043491, 0.006880229, -0.0034436246, -0.008284357, -0.0058411737, 0.0039456007, 0.0010513411, -0.017383108, -0.0049390215, 0.013444529, 0.0012110607, -0.03709707, 0.03959642, 0.0011250578, -0.012981166, 0.041646447, -0.021286584, -0.033530585, -0.01280565, -0.029065456, -0.015950898, 0.029570943, -0.0038227392, 0.02264859, 0.011071552, -0.012847774, 0.009534031, -0.010229074, 0.041169044, 0.029711355, -0.02023349, 0.016681043, 0.06874612, -0.0022132574, 0.023561273, 0.014420398, -0.0047986084, 0.032014124, -0.011850843, 0.008115862, 0.005609493, 0.017706059, -0.016358094, -0.046336234, -0.011226006, 0.004215895, 0.032547694, 0.04330332, 0.02385614, 0.009470846, -0.008024593, 0.0024045696, 0.03176138, 0.01717249, 0.010966242, -0.043499894, 0.04046698, 0.035917602, 0.0055498173, -0.0010732806, -0.012770547, -0.00039951838, 0.021932485, -0.02746475, 0.032968935, 0.0155998655, 0.019306764, -0.041646447, 0.00061869406, 0.012882877, 0.01181574, 0.0068381047, -0.007940345, -0.04552184, -0.0049706143, 0.017214613, -0.0065151555, 0.00069767627, 0.038332704, -0.017355027, 0.0053883423, 0.026074663, -0.0076173963, -0.020935554, 0.0076244166, -0.025091773, 0.03510321, -0.010762643, 0.013086475, 0.008024593, 0.0062237987, 0.025260268, -0.0045739478, -0.054704838, -0.025330475, -0.0011847332, 0.004889877, 0.0020868857, 0.007568252, 0.038613528, -0.006490583, 0.015178627, 0.012447597, -0.030413419, -0.034513474, -0.019742044, -0.0013154927, -0.010004414, 0.028251061, -0.027843865, -0.008494976, 0.008740699, 0.013613024, 0.04378072, 0.022971539, -0.027380502, -0.015094379, -0.016442342, -0.019952664, 0.0011899987, 0.04891983, -0.005044331, 0.017663933, -0.0060272207, 0.020275613, 0.04423004, -0.023701686, 0.004917959, -0.036395006, 0.02854593, 0.045774583, 0.02549897, 0.009175979, -0.0022922396, 0.045128684, 0.01279863, -0.038585447, 0.011731492, -0.00787716, -0.0071259513, 0.0001284558, 0.027057553, -0.0044897003, -0.016512549, 0.032660026, -0.0033190083, 0.021090006, -0.006862677, -0.010818808, 0.004738933, -0.03619843, 0.0105028795, -0.012482701, -0.014925884, 0.0011724471, -0.032856602, -0.011127717, 0.004236957, 0.0013742906, -0.011914029, 0.006227309, -0.010011435, -0.0034857485, -0.0119561525, 0.025091773, 0.01958759, 0.0060202, -0.0042966325, 0.006750347, 0.014841637, -0.0038683736, 0.015192668, -0.0347943, -0.0041316478, 0.008551141, 0.024136966, -0.017663933, 0.042741667, 0.021104049, -0.0011794678, -0.05175617, 0.0025748203, -0.01784647, -0.0247267, 0.0053462186, -0.02627124, 0.030020263, -0.02363148, 0.004619582, -0.037518308, -0.03423265, -0.021581452, -0.017018035, 0.0029188318, -0.0028100119, -0.011085593, 0.0177201, 0.020486232, -0.030694244, -0.008621348, 0.0028328288, -0.012138689, 0.0032136987, -0.0081650065, -0.010776685, 0.019826291, -0.013858747, -0.024122924, 0.028840795, -0.028840795, -0.01729886, -0.02287325, -0.011914029, 0.0061781644, 0.031705216, 0.005504183, 0.0023133014, 0.033418253, 0.01083987, -0.0062975157, 0.011654265, 0.031031236, 0.025400681, 0.07531744, -0.029233951, -0.019728003, -0.018520452, -0.010629252, 0.037602555, -0.004103565, -0.020191366, -0.032126456, 0.030918906, -0.0053637703, 0.016063228, 0.012595031, -0.012496742, 0.01083285, 0.022437971, -0.0058271326, -0.0026186993, 0.011099635, 0.004738933, 0.00025362067, 0.06874612, -0.010594148, 0.010636272, 0.003921028, 0.01979821, 0.0067433263, 0.026678437, 0.0026976815, -0.0009925432, -0.0118578635, -0.027450709, -0.012117627, 0.004430025, -0.020191366, 0.028040443, -0.011408542, -0.007940345, -0.03777105, 0.012960104, 0.004991676, 0.027857905, -0.00284687, 0.0006230819, 0.004114096, -0.006996069, 0.027197964, -0.010362467, 0.0047108503, -0.007066276, -0.01871703, 0.011352377, -0.00678896, -0.01641426, 0.022437971, -0.03381141, -0.0066590784, -0.0012057952, 0.0037068988, -0.021806113, -0.011801698, 0.014869719, -0.019531425, -0.0021500716, 0.01937697, -0.041871108, 0.0029083008, 0.0075471895, 0.010348425, 0.00037538493, -0.02330853, -0.029964099, 0.0019025939, -0.0062764534, 0.0015164586, 0.015824527, -0.013879809, -0.003963152, 0.0039982554, 0.022887291, -0.011113675, -0.030441502, -0.03271619, 0.0008898663, -0.009814857, -0.01871703, 0.03830462, -0.015684113, -0.036451172, 0.036366925, -0.022030773, 0.017930718, 0.032351118, 0.024010595, 0.010025476, -0.02780174, -0.018408122, 0.011548955, 0.0027661328, -0.0032259848, 0.003283905, 0.0008446709, -0.036254592, 0.014869719, -0.00024528368, -0.0050408207, -0.038894355, -0.0005050474, 0.012573969, -0.0090987515, 0.01728482, -0.002362446, 0.0058797873, 0.0053532394, -0.0052373987, 0.025147937, -0.002602903, 0.032772355, -0.018983815, -0.040129986, 0.00459501, 0.00415973, 0.008572203, -0.020851305, 0.028616136, 0.017748183, 0.025442805, 0.0016507284, -0.0044897003, -0.009049607, -0.002057048, 0.034653887, 0.023069829, -0.0013032065, 0.00295569, -0.0015840322, 0.012398453, -0.005026779, -0.020710893, 0.02680481, 0.018983815, 0.005511204, -0.0051917643, -0.023729768, 0.015698155, -0.014181696, 0.04192727, 0.0067538572, -0.014687182, -0.0004686278, -0.0071961577, 0.0062343297, 0.008635389, 0.003643713, 0.015080338, -0.017958801, -0.023505108, -0.009920167, -0.0028082565, 0.0077156853, 0.03339017, 0.0050338, -0.032519612, -0.005514714, 0.02999218, 0.0069644763, -0.026650356, 0.017144406, 0.0076103755, -0.011176862, -0.003152268, -0.003963152, -0.020458149, 0.016540632, -0.026734604, -0.046111573, -0.0002913566, 0.03774297, 0.0012075504, 0.023996552, -0.038922437, -0.0082632955, 0.0006059691, -0.014083407, -0.043190986, 0.0032154538, -0.0033628873, 0.00039096197, -0.0037384916, -0.0055638584, -0.016119393, -0.06908311, 0.011296213, -0.007582293, 0.026313365, 0.020373901, -0.021047883, 0.043752637, 0.013992138, -0.018478328, -0.004521293, 0.001793774, 0.013641107, -0.008347543, -0.017228654, 0.031620968, 0.0009758692, -0.014258923, -0.009428722, 0.008572203, 0.0012391432, -0.009583176, 0.003238271, 0.010846891, 0.010215034, -0.0074137975, 0.006630996, 0.01138046, 0.014813554, -0.02066877, -0.015670072, -0.015908774, 0.01628789, -0.0042509986, -0.035749108, 0.020753017, 0.0011066287, -0.002767888, 0.03793955, 0.0033997456, -0.013198806, -0.04715063, -0.00711191, 0.01805709, -0.010874974, 0.03010451, -0.05231782, -0.023336612, -0.0001772712, 0.028293185, -0.0030346722, 0.012180813, -0.019952664, 0.02527431, 0.0034436246, 0.025260268, 0.0005550695, -0.0413937, 0.0023290978, 0.017340984, 0.030722328, 0.029065456, -0.03875394, -0.017677976, -0.0072523225, -0.038360786, 0.012644175, 0.025541093, -0.0042545088, -0.0013637596, -0.060096692, 0.0037033884, 0.023603396, -0.02702947, 0.005507693, 0.023125993, -0.007947367, 0.028026402, -0.008600286, -0.0010855667, -0.0015252344, -0.00010476114, -0.03403607, -0.029964099, 0.04257317, 0.017986884, -0.00042277426, 0.007399756, -0.022381805, -0.0310874, 0.014392315, -0.018211544, 0.0020008828, -0.016456384, 0.010671375, -0.01390087, 0.009821878, 0.025344515, -0.022943456, 0.00057832536, -0.0020605584, 0.03184563, 0.016344054, 0.0002696365, -0.022901332, 0.009070669, 0.005974566, 0.0039771935, 0.0061114687, 0.015262875, 0.016512549, -0.025133897, 0.027015429, -0.004795098, 0.020458149, 0.035018962, -0.0067678983, 0.04257317, -0.02353319, -0.0060482826, 0.022030773, -0.012917981, -0.020107117, 0.01488376, -0.001979821, -0.0030487136, -0.017467355, -0.01082583, -0.009161937, -0.0003808698, -0.012454618, -0.0018762664, 0.004408963, 0.020724934, -0.0069012907, 0.003380439, -0.003338315, 0.04869517, -0.00678194, 0.012861815, 0.00017496756, 0.045465674, 0.026369529, 0.0035541998, 0.007069786, -0.014125531, -0.01432913, -0.014588893, 0.01784647, -0.0068240636, -0.013261992, 0.00919002, 0.004236957, -0.020458149, -0.0046581957, 0.02954286, -0.032014124, -0.0068521462, -0.0071259513, 0.0038613528, -0.0075471895, -0.00078675064, 0.02111809, -0.019433135, 0.012770547, 0.030301088, 0.026186993, 0.00831946, 0.018323874, -0.05318838, -0.02210098, -0.018745113, 0.016456384, 0.021918442, 0.019475259, -0.00087801897, -0.0085090175, -0.054761004, -0.016456384, 0.0060693445, -0.007280405, 0.027506873, 0.044904023, 0.009351495, -0.012707361, 0.0023940387, -0.019025939, -0.028194897, 0.025527053, -0.036984738, 0.035861436, -0.016891664, -0.009934207, -0.02176399, -0.013767478, -0.024684576, -0.015571783, 0.006097427, 0.0054620593], "content_hash": "89478f3781f96fb856690c0490fe95fc"}, {"content": "# GPT-2 Overview\n\n## What Is GPT-2?\n\nGPT-2 (Generative Pre-trained Transformer 2) is a language model created by OpenAI in 2019. It was one of the first models to demonstrate that scaling up transformers could produce impressively fluent text. It remains one of the most well-studied and accessible models for learning about transformer internals.\n\n## Architecture Details\n\n| Property | Value |\n|----------|-------|\n| Parameters | ~124 million (small variant) |\n| Layers | 12 |\n| Attention Heads | 12 per layer (144 total) |\n| Hidden Dimension | 768 |\n| MLP Dimension | 3072 (4x hidden) |\n| Vocabulary Size | 50,257 tokens |\n| Positional Encoding | Learned absolute positions |\n| Max Sequence Length | 1024 tokens |\n| Normalization | LayerNorm |\n| Activation Function | GELU |\n\n## Why Start with GPT-2?\n\nGPT-2 small is the **recommended starting model** for learning with this dashboard:\n\n- **Fast**: Small enough to load quickly and run interactively\n- **Well-studied**: More research papers have analyzed GPT-2's internals than almost any other model. Many examples and references use GPT-2.\n- **Clear patterns**: With 12 heads per layer, the attention patterns are easy to visualize and categorize\n- **Manageable size**: 144 total attention heads is small enough to explore systematically\n\n## What to Expect in the Dashboard\n\nWhen analyzing GPT-2, you'll typically see:", "source_file": "gpt2_overview.md", "chunk_index": 0, "embedding": [-0.05763732, 0.011029581, 0.042829685, -0.01631886, 0.021965442, -0.01574483, -0.03505099, 0.03502756, 0.0048851143, 0.061011214, 0.01254666, -0.028490642, -0.04006497, -0.033036023, 0.004709391, -0.042431377, -0.022398893, -0.022469183, 0.019575601, 0.038143724, 0.014561623, -0.034324665, -0.023582099, 0.03345776, 0.013800155, -0.01165047, -0.00070326024, 0.028256346, -0.009295776, -0.01937645, 0.0043433, -0.02647568, -0.057309303, -0.049764905, 0.0063670496, -0.013612716, 0.027366012, 0.048218537, -0.0019358876, 0.054263428, 0.027319152, -0.005957028, -0.026873987, 0.052904498, -0.033809207, -0.0005128931, 0.014245321, -0.011351741, -0.054450866, 0.04817168, 0.00039647624, 0.048640274, -0.033293754, -0.0015112225, -0.021766288, -0.012710669, -0.033410903, -0.016482867, -0.025140181, -0.025866505, -0.01319098, 0.02757688, 0.022082591, 0.037065953, 0.01629543, -0.013940734, -0.06115179, 0.05515376, -0.03390293, -0.005450358, 0.021016534, 0.0014453261, -0.03704252, -0.021379696, 0.008879896, -0.0053097797, -0.04069757, 0.051170696, -0.025796216, -0.025023032, -0.0056290105, -0.027998617, -0.03760484, -0.029849572, -0.009114194, -0.0043110843, -0.014186746, -0.014948215, -0.0066774944, -0.048312258, -0.0053097797, -0.0124646565, -0.026616259, 0.0037926997, 0.025843075, -0.0016913391, 0.023113502, -0.017853511, -0.01808781, 0.04327485, 0.023617243, 0.037979715, -0.01966932, 0.0032479567, 0.0069235074, -0.0023298012, 0.0056495117, -0.006841503, -0.05435715, -0.049015153, -0.09498443, 0.026639689, 0.029638704, 0.015463672, 0.018743845, -0.0073686736, -0.0129449675, 0.02417956, 0.017818367, -0.051030114, 0.0168226, -0.02635853, 0.0021906868, -0.00020025161, -0.010590272, 0.020301927, -0.0323097, 0.010361832, -0.00091010146, 0.016096275, -0.031677097, 0.047468785, 0.007485823, -0.043907456, -0.018696984, 0.0054181423, -0.049952343, 0.022410609, -0.026171092, 0.0040328554, -0.002530419, -0.036480207, 0.02219974, 0.056559548, -0.095640466, -0.009348492, -0.019821616, 0.047937382, -0.08219176, 0.05543492, 0.0016649805, 0.050795816, -0.019915335, -0.04109588, -0.025960224, -0.040721, 0.014456189, 0.03830773, 0.024671584, -0.04224394, 0.0045629544, 0.0026329244, -0.003681408, 0.007233952, -0.04554754, 0.05871509, -0.025632206, -0.009565217, -0.062932454, -0.06237014, 0.034207515, 0.018966427, -0.020126203, -0.023172077, 0.023535239, 0.0064314813, -0.08720573, -0.009196199, -0.032098833, -0.05482574, -0.025819644, -0.022024017, -0.03376235, -0.009541788, -0.046344154, 0.0019988553, -0.016400864, 0.0276706, 0.017115472, 0.0075443974, 0.023406375, 0.013436994, 0.07684976, 0.022504328, -0.010209537, -0.033129744, 0.06743098, 0.0046888897, 0.0069235074, 0.03713624, 0.016166566, 0.0065369154, 0.019774755, -0.060776915, -0.0045834556, -0.01663516, 0.010894859, 0.011369313, -0.043673158, -0.028303204, 0.013671291, -0.01631886, -0.035824172, -0.010976864, -0.045219522, -0.017806651, -0.04910887, 0.029755853, 0.037768845, 0.02040736, 0.0074038184, 0.053654253, 0.022668336, 0.004961261, -0.024999602, 0.015592535, 0.01749035, -0.025468199, 0.012780959, -0.0015492958, 0.02640539, -0.039924387, -0.013284699, -0.0135189975, -0.0012300648, -0.022363748, 0.04437605, 0.031794246, -0.011140872, -0.017771507, 0.004120717, 0.01436247, -0.022996353, 0.010361832, 0.06213584, -0.016002556, -0.051030114, -0.07057057, 0.06780586, 0.037839133, 0.0035320432, -0.015920553, -0.04093187, 0.025725925, -0.02337123, -0.0012988899, 0.0041119307, 0.016131422, -0.017244337, -0.012441226, 0.008557736, 0.020887671, -0.026991135, -0.000904244, -0.03378578, -0.016682021, -0.030622754, -0.029334115, -0.04812482, -0.051217552, 0.04224394, -0.0077025485, -0.0049378313, 0.036597356, -0.06237014, -0.024695015, 0.014713918, 0.012136639, -0.012101495, 0.03950265, 0.00060807663, -0.0138118705, 0.0480311, 0.039877526, 0.010414548, -0.004454592, 0.0029711921, -0.010613701, 0.006138609, -0.024484146, 0.023008069, -0.0034412525, 0.027764319, -0.0198099, -0.020583084, 0.020126203, 0.013612716, 0.030763334, 0.01013339, -0.011357598, -0.007749408, -0.05037408, 0.017654357, -0.002753002, 0.015393382, 0.042595387, 0.0069586523, 0.050327223, -0.010490695, 0.027178574, -0.02769403, -0.028514072, 0.0026182807, 0.016623447, 0.022832345, 0.036855083, -0.002017892, -0.014526479, 0.006191326, -0.04090844, -0.03711281, -0.035683595, 0.014913071, 0.00023869114, 0.025444768, 0.038659178, -0.00420565, 0.010279827, 0.026709978, -0.0074741077, 0.033223465, -0.015147369, -0.04097873, 0.014795922, -0.022422323, 0.0052482765, 0.0050432654, 0.019013287, -0.02870151, 0.028350065, -0.030646184, -0.019294444, -0.03376235, -0.0010828963, -0.057262443, 0.057262443, 0.007848985, 0.035683595, 0.04224394, -0.021543706, 0.0105727, -0.025116751, -0.015662825, 0.018696984, 0.020020768, 0.026264811, 0.01087143, -0.0055967947, 0.02525733, 0.0069235074, -0.033996645, -0.007848985, 0.020559654, 0.009547645, -0.05145185, 0.020653373, -0.018896138, 0.03823744, -0.011919913, -0.004929045, 0.01560425, -0.001041162, -0.0025626349, 0.0036228336, -0.03950265, -0.002753002, -0.003915706, 0.008704172, -0.02886552, 0.023710962, 0.023664104, 0.034957267, 0.0031161641, -0.0036404058, -0.0015463671, 0.043509148, 0.017279482, -0.007151948, -0.026171092, 0.037839133, 0.008792034, -0.007409676, -0.029334115, 0.018099524, -0.015498816, 0.034207515, -0.011750047, 0.00034632182, 0.04322799, 0.047398496, -0.011492319, -0.010713278, -0.03003701, -0.023535239, 0.010865572, -0.011299023, 0.0006110054, 0.057590462, 0.022375464, -0.0055323625, 0.054966323, -0.0076674037, -0.00721638, -0.0051721293, 0.011568466, 0.008153572, 0.01443276, 0.042267367, -0.01199606, 0.034207515, -0.01261695, -0.001707447, -0.047421925, -0.00055243087, -0.052201606, 0.01615485, 0.02999015, 0.00093865657, -0.038729466, -0.025796216, 0.0067653563, 0.008012993, 0.026686547, -0.042689104, 0.008891611, 0.00059123646, 0.03711281, 0.027881468, 0.0033680345, 0.023218937, 0.022070875, -0.0385186, -0.004577598, -0.023710962, -0.035004128, -0.005066695, -0.028420353, -0.046062995, -0.007620544, -0.025140181, 0.030083869, 0.06743098, 0.001723555, -0.025304189, -0.003895205, -0.03711281, 0.03856546, 0.033364043, 0.074928515, 0.0015815118, 0.035683595, -0.008633883, -0.022656621, -0.037065953, -0.01562768, -0.0012966932, 0.021895153, 0.0023839828, -0.007866557, -0.025889935, 0.03127879, -0.020501079, 0.014444475, -0.002086717, -0.013647862, 0.02406241, 0.03104449, 0.001259352, -0.01914215, -0.041986212, 0.02229346, 0.032333132, 0.01794723, 0.024577865, -0.01268724, 0.09123566, -0.036691073, 0.011515749, 0.032145694, -0.017103758, -0.023874972, -0.025819644, 0.044001173, -0.010379404, -0.008874038, 0.00011147462, -0.020372216, -0.0721638, -0.027272293, 0.021203974, 0.026663117, -0.023535239, -0.008926756, 0.060448896, 0.024812164, 0.0030341598, 0.015323092, -0.008012993, -0.006823931, 0.009237201, -0.05140499, 0.0005015443, -0.0072808117, -0.0019842116, -0.008674885, 0.035871033, 0.04053356, 0.028092336, 0.024390427, 0.011966773, 0.00082004315, 0.017548924, -0.03943236, -0.04423547, -0.0038688465, -0.014081312, -0.027764319, -0.061714105, -0.0287718, -0.016881175, 0.025140181, -0.013577572, 0.0011356133, -0.0014416652, -0.087908626, 0.021403126, 0.011632898, -0.028350065, -0.049999204, 0.009746799, 0.009377779, 0.02509332, 0.005040337, -0.014526479, 0.006402194, 0.018860994, 0.011439602, -0.00983466, 0.009518358, 0.059886582, 0.06026146, -0.0314428, 0.03966666, -0.010514125, -0.031841107, -0.031723958, 0.015979126, -0.005283421, 0.0023708034, -0.009887378, 0.010748423, 0.006946937, 0.03624591, -0.017115472, -0.0018099524, 0.020594798, 0.009237201, -0.014667057, 0.011574323, -0.020723663, -0.0008793499, 0.00017984206, 0.009729226, -0.013390133, 0.0005582883, -0.019868474, 0.008428873, 0.021836579, 0.0046947473, 0.030786764, -0.03219255, -0.023722678, 0.0005227775, 0.035589874, -0.028584363, 0.029498124, 0.006097607, 0.008270721, -0.009869806, 0.037347108, 0.043720014, -0.01691632, -0.00074279803, 0.03116164, -0.023664104, -0.02392183, 0.035425864, -0.019505313, -0.055997234, 0.014116458, -0.003403179, -0.0057900907, -0.037745416, -0.013741581, -0.006613062, 0.0077259783, -0.010209537, 0.01806438, -0.020501079, -0.006777071, -0.0072280946, 0.0002441825, -0.004463378, -0.0138353, -0.053326234, 0.01620171, -0.016307143, -0.0010272505, 0.042782824, 0.002118933, 0.038659178, -0.026311671, -0.015803404, -0.029873, -0.0062499004, 0.0057490882, -0.01615485, -0.0143859, -0.020313641, 0.016834315, -0.01686946, 0.020114487, 0.094562694, 0.018931283, -0.0066247773, 0.009506644, 0.020114487, -0.008926756, 0.018146385, -0.020969676, -0.012675525, -0.018966427, -0.014585054, -0.015194229, 0.0005473056, -0.010918289, -0.020747092, 0.022117736, -0.021192258, -0.0044223755, -0.03612876, -0.025960224, 0.054685164, 0.0056085098, 0.012417797, 0.00393035, 0.00014222624, -0.03479326, -0.025749356, 0.040205546, -0.058105916, 0.008071568, -0.010467266, -0.0312085, -0.014081312, 0.02741287, 0.029544985, 0.011433745, -0.00077684445, -0.024296708, -0.03610533, 0.0063787643, -0.0078607, -0.0115040345, -0.0043696584, -0.014889641, -0.008270721, -0.051311273, 0.00930749, 0.031700525, 0.006244043, 0.010783568, -0.041306745, 0.031841107, 0.020208206, 0.010068959, -0.008669028, -0.0033270323, 0.03125536, -0.0012534945, -0.01995048, -0.020747092, 0.022469183, -0.003915706, -0.024390427, -0.015733114, -0.0034236803, 0.06602519, -0.011281451, -0.057215583, -0.03950265, 0.0034236803, -0.011732475, 0.021332838, -0.0052219178, 0.03327032, -0.019657606, 0.035683595, 0.0055762934, -0.03603504, -0.025491627, 0.008844751, 0.0059746, 0.007737693, -0.017712932, 0.005037408, -0.011263879, -0.0016195853, 0.0052658487, 0.0041470756, -0.026311671, 0.028045477, 0.01562768, 0.019200725, 0.005582151, -0.02989643, 0.009781944, -0.010555127, -0.04943689, 0.0075502545, 0.021215688, 0.009664794, -0.008587023, -0.014948215, 0.0033592484, 0.020817382, -0.011632898, 0.008704172, -0.035496153, -0.007907559, 0.0002465621, 0.027084855, -0.013753296, 0.013272985, -0.008838894, -0.010625416, 0.0020940388, 0.022082591, 0.008546022, -0.057121865, -0.002477702, 0.0077084056, -0.004738678, -0.003212812, 0.004741607, 0.03373892, 0.009125909, 0.008341011, -0.015264518, 0.023511808, 0.01507708, -0.019610746, -0.007140233, 0.018216673, -0.019868474, -0.03015416, 0.004952475, 0.020899385, -0.0072105224, 0.0108245695, 0.003491041, -0.007848985, 0.016389148, -0.017080327, 0.0074565355, -0.01050241, 0.024577865, -0.02284406, 0.031091351, 0.018896138, -0.018837564, 0.013952449, 0.014749062, 0.008264864, -0.014807637, -0.012874678, -0.013729866, 0.008979472, -0.014339041, 0.006525201, -0.022281744, 0.014327326, 0.030997632, 0.014128173, -0.008920899, -0.023277512, -0.011550894, -0.10159163, -0.02872494, -0.016049417, -0.02521047, 0.00344711, -0.008680743, 0.017701218, -0.014737347, 0.0026065658, -0.051873587, 0.0030195161, -0.046414442, 0.0055352915, 0.0071343756, 0.028068906, -0.042618815, 0.003704838, 0.0038600604, 0.046227004, -0.030224448, -0.014866211, 0.003040017, 0.013729866, -0.0059072394, -0.03509785, -0.008282436, 0.024413858, 0.007661546, 0.0023254082, 0.008727603, -0.038050003, 0.049483746, 0.0067477836, 0.02038393, -0.017677788, 0.01620171, -0.020126203, -0.023804681, 0.03355148, 0.016236855, -0.003022445, -0.025140181, -0.01326127, -0.0022390108, 0.055762935, -0.008786177, 0.036152188, -0.0072222375, -0.014350755, 0.01985676, 0.034957267, 0.02640539, 0.004026998, 0.026194522, 0.013472138, 0.00415879, 0.013495567, -0.048218537, 0.038096864, 0.006413909, 0.0063026175, 0.03966666, 0.015779974, 0.0032303843, -0.002555313, 0.009787801, 0.023230651, 0.021801433, -0.024366997, -0.018029235, -0.028795231, 0.010560985, 0.004855827, -0.030575896, -0.019622462, 0.010625416, 0.009881521, 0.044563487, -0.009448069, 0.0017499136, 0.008428873, -0.0011407386, -0.016658591, 0.0073511014, 0.044212043, 0.028232915, -0.00602146, -0.007679119, 0.012488086, 0.008118427, 0.02640539, 0.0051926305, -0.008007136, 0.0030722332, -0.0040211403, -0.0024235204, 0.019645892, 0.03983067, 0.034722973, -0.025796216, 0.059933443, 0.010578557, -0.05899625, -0.01691632, -0.0139055895, 0.006349477, 0.006777071, 0.0036872656, -0.029662132, -0.031677097, -0.007825555, 0.014268751, 0.0057490882, 0.024929313, -0.027623741, 0.035871033, 0.08069225, 0.014327326, 0.017642643, -0.0068005007, -0.004832397, -0.005564579, 0.012324077, 0.02160228, -0.04721106, 0.0018202029, 0.016494583, 0.07150777, 0.0030575895, -0.00037194815, 0.008762747, 0.015569106, 0.006308475, -0.0276706, -0.001787987, 0.0062381853, 0.021684285, -0.007679119, -0.005869166, 0.012007775, -0.033246893, 0.003180596, -0.013319844, -0.008405442, -0.0052482765, -0.027740888, -0.0004612743, -0.05145185, 0.0027544664, 0.004577598, 0.05407599, 0.042314228, -0.017759793, 0.051873587, 0.007737693, -0.0038454167, -0.026709978, -0.019997338, -0.03622248, 0.0024264492, 0.031419367, 0.03125536, 0.03394979, -0.00086543843, 0.021039965, 0.011568466, -0.009155196, 0.026920846, -0.015147369, -0.024320137, -0.030318167, 0.050092924, 0.040299263, 0.032098833, 0.011275593, 0.004823611, -0.0066774944, 0.017818367, -0.00031428886, 0.015873693, 0.004437019, 0.03132565, 0.02171943, 0.012628665, 0.012300648, -0.025819644, 0.012488086, -0.017092044, -0.039104346, 0.03493384, -0.008276579, 0.011509892, -0.012581806, 0.014467904, 0.00027310365, -0.02654597, 0.029474694, -0.006027317, -0.019458452, -0.006220613, 3.9537797e-05, 0.013530713, 0.0038454167, -0.026147662, -0.0105434125, 0.04444634, 0.011773477, -0.00075780775, -0.00926063, -0.024531005, 0.005339067, 0.023910116, 0.0017015896, -0.012851248, 0.0013296414, -0.0014951144, 0.022082591, 0.010789425, -0.026077373, -0.02215288, -0.07103917, -0.023792967, -0.011738332, -0.028443784, -0.0076556886, 0.032614287, -0.01849783, 0.028115766, -0.00077391573, 0.013155836, 0.0038922764, -0.011685615, 0.008774462, 0.010338401, 0.018427541, 0.03357491, 0.017185763, 0.008786177, 0.0050022635, 0.0029199393, 0.029708993, 0.008440588, 0.0021848292, -0.0054708594, -0.021977156, -0.0116739, 0.027342582, 0.0054152138, -0.009711654, -0.017279482, 0.0116739, 0.022410609, 0.0110237235, -0.016881175, 0.01562768, 0.030435316, 0.024366997, 0.019763041, 0.011410315, 0.0013970021, 0.009477356, 0.054966323, 0.03573045, -0.016693735, 0.021778004, -0.004961261, -0.015943982, -0.00843473, 0.040439844, 0.029404406, -0.030388458, 0.023101788, -0.010139248, -0.011439602, 0.007913416, -0.02870151, 0.028209485, 0.0021980086, 0.026709978, 0.003063447, -0.025585348, -0.009547645, 0.010836285, -0.020137917, -0.006525201, 0.008147715, -0.03484012, -0.0048821857, -0.009178626, -0.016928034, -0.009951809, 0.010531697, 0.0067594987, -0.023828112, -0.013331559, 0.008329296, 0.0056319395, -0.009711654, -0.0012783888, -0.032731436, 0.0054591447, -0.009061477, 0.02649911, -0.048687134, -0.004234937, 0.02509332, 0.016904604, -0.044774357, 0.013741581, 0.026709978, -0.026827127, -0.028279774, -0.009776087, 0.03605847, 0.038823187, 0.030365027, -0.011855481, -0.029544985, 0.010719135, -0.0111174425, 0.0078431275, 0.00037890387, 0.005295136, -0.012370937, 0.0019871404, -0.02171943, 0.021356268, 0.02399212, 0.019388163, 0.00049349025, 0.028514072, 0.0119784875, 0.0009855161, 0.02157885, -0.04683618, -0.030271308, -0.008200432, -0.03130222, 0.019470168, -0.04458692, 0.016810885, -0.0014702203, -0.036363058, -0.019517027, 0.009992812, 0.0041031446, 0.018427541, -0.025140181, 0.021473417, 0.023218937, 0.047468785, -0.03123193, -0.034348093, 0.0051838444, -0.013952449, 0.035285287, -0.022410609, -0.0039537796, -0.013026971, 0.0038073433, 0.020665089, 0.010162678, -0.024109269, -0.007889987, 0.032005113, 0.0039039913, -0.02406241, -0.021883437, 0.035168137, 0.00276911, 0.014678773, 0.024976172, -0.03622248, 0.0025040605, -0.0029375118, 0.021215688, 0.028584363, -0.024132699, 0.04444634, 0.02109854, -0.012406082, -0.07539711, -0.0033826781, 0.039924387, -0.006946937, 0.019165581, 0.002590458, 0.006179611, -0.05304508, 0.0062499004, 0.0064959135, -0.021871723, 0.007503395, -0.017010039, -0.021496845, 0.009155196, 0.0099635245, -0.0044135894, 0.017302912, -0.03017759, -0.023125218, 0.0020471793, -0.01620171, -0.014854496, 0.033082884, -0.03954951, 0.016283715, 0.02098139, -0.0052541336, -0.0025816716, -0.04566469, 0.017701218, -0.024390427, 0.00063150644, -0.010285684, -0.022972925, -0.020231636, 0.013015256, -0.0029316545, -0.0011070583, -0.009500786, -0.010186108, -0.0040855724, -0.018275248, 0.034113795, 0.0028584362, -0.0043608723, -0.01312069, -0.0045717405, 0.015885407, -0.031770814, -0.006507628, -0.007005512, 0.014901356, -0.008704172, -0.020946246, 0.011779334, -0.051076975, -0.050936397, 0.030739903, 0.0138118705, 0.022609761, 0.024812164, 0.03994782, 0.009325062, 0.008774462, -0.02649911, -0.0074331057, 0.008235577, 0.013647862, 0.012921537, 0.0076674037, -0.0173732, -0.005614367, -0.009471498, -0.0023180863, -0.0023927689, -0.0013098725, 0.007128518, -0.0068473606, -0.0042407946, 0.005476717, 0.027740888, 0.008762747, 0.020208206, 0.010971006, 0.006349477, 0.06977396, 0.0063201897, -0.036386486, 0.042665675, -0.0054708594, 0.029170107, -0.00039940496, -0.0031952397, -0.006308475, 0.02640539, -0.011662185, -0.051592432, -0.0047035334, -0.008218004, 0.008024708, 0.01266381, -0.03505099, 0.0101568205, 0.042829685, 0.015088795, -0.011006151, -0.028959239, 0.0029873, 0.014011024, -0.010191965, 0.013729866, -0.0035144708, 0.028467214, 0.0056553693, 0.011837909, -0.024788734, -0.037885994, -0.018146385, -0.022434039, -0.019165581, 0.0028906523, -0.0015287948, -0.017279482, -0.06358849, -0.015920553, -0.008967758, -0.004735749, 0.0033768206, -0.0070113693, 0.032005113, -0.012511516, 0.021590564, -0.005037408, 8.168399e-05, 0.014760777, 0.023886686, -0.0116739, 0.028350065, 0.014467904, -0.0032743153, 0.002821827, 0.02222317, 0.03357491, 0.0013530712, 0.029568413, -0.0016576587, -0.0347464, 0.02348838, -0.001331838, 0.018708698, -0.04076786, -0.029662132, 3.592265e-05, 0.03458239, -0.0048411833, -0.0027632527, 0.0025582418, 0.0054620733, -0.019306159, -0.058152776, -0.0066657793, 0.021215688, -0.026733408, 0.007040656, 0.0051486995, 0.01500679, 0.0131324055, -0.025772786, -0.017724648, -0.019927049, -0.009711654, -0.0037956284, -0.036644217, 0.0358476, -0.013659576, 0.00060807663, 0.0061971834, 0.00012556912, -0.016810885, -0.028420353, -0.03994782, 0.010291542, 0.042853113, -0.02098139, -0.016541442, -0.023359515, 0.007837269, -0.0064841984, 0.013647862, -0.008868181, 0.022410609, -0.007907559, -0.0056290105, 0.008153572, -0.030435316, -0.021731144, 0.002001784, 0.012441226, 0.004507309, -0.029380975, 0.0469299, 0.017748076, -0.021543706, -0.0041353605, -0.0071870927, -0.0046068854, 0.0075092525, 0.009073192, -0.02420299, 0.029427836, -0.00983466, 0.01937645, -0.0062088985, 0.041447327, -2.193112e-05, 0.00018597407, -0.062276423, 0.00753854, 0.010385261, 0.00043601403, -0.001799702, -0.022445753, -0.043579437, -0.008628026, -0.041822203, -0.008967758, 0.022562902, 0.00452781, -0.037698556, -0.016025987, 0.008610453, -0.016014272, 0.009852233, 0.029380975, -0.00093499565, -0.040228974, -0.01749035, 0.019610746, 0.0066657793, -0.03840145, 0.0034207515, -0.0012688704, -0.0035701166, 0.031630237, -0.01087143, 0.009202056, 0.003652121, -0.010039671, -0.006413909, 0.0004993477, -0.0017616284, -0.020337071, 0.032544, -0.0005370551, 0.0069059352, -0.007966134, 0.032614287, 0.009489071, -0.051967308, 0.03017759, -0.0068005007, -0.007040656, 0.015791688, 0.020032484, -0.017478634, -0.009940094, -0.020829096, 0.06668123, -0.000464203, 0.0047679655, 0.023230651, -0.010115818, 0.023617243, 0.0030605183, -0.013296414, 0.019036716, -0.027811179, -0.02052451, -0.009254773, -0.032661147, -0.01326127, -0.018907852, -0.009887378, -0.0012344578, -0.0029155463, 0.00962965, 0.00095110363, 0.048265398, -0.008920899, 0.010660561, 0.017244337, 0.0010880216, 0.08294151, 0.01087143, 0.013026971, 0.04573498, 0.018860994, -0.0022961209, 0.02346495, 0.023582099, 0.05651269, 0.028326634, -0.0027793606, -0.016436009, -0.015955698, 0.03722996, 0.05201417, 0.0029536197, -0.014924786, -0.022375464, 0.006718497, -0.01500679, -0.012839533, 0.019107006, 0.010279827, -0.037347108, -0.0034734686, -0.0036872656, 0.027459731, 0.01734977, -0.005066695, 0.005344924, 0.014104743, 0.036456775, 0.0022258314, -0.020079343, -0.031911395, -0.029708993, -0.058152776, 0.00040782505, 0.015569106, -0.007731836, -0.011925771, 0.02874837, 0.009412924, -0.040111825, -0.006138609, -0.03837802, -0.0013494103, 0.015979126, 0.037464257, -0.0074448204, -0.012066349, 0.021590564, -0.025679067, -0.018240103, 0.027436301, 0.016471153, 0.019763041, -0.023933547, -0.030927343, 0.0017381987, 0.012827818, -0.013987593, -0.013272985, 0.021344552, 0.016951464], "content_hash": "684ce36d285d76b89e0dcb331f6dca24"}, {"content": "model. Many examples and references use GPT-2. - **Clear patterns**: With 12 heads per layer, the attention patterns are easy to visualize and categorize - **Manageable size**: 144 total attention heads is small enough to explore systematically ## What to Expect in the Dashboard When analyzing GPT-2, you'll typically see:\n\n- **Tokenization**: GPT-2 uses BPE (Byte-Pair Encoding). Common words are single tokens; rare words get split. Spaces are typically attached to the beginning of the following token.\n- **Embeddings**: 768-dimensional vectors, which capture rich semantic information despite being relatively compact.\n- **Attention patterns**: You'll see a good mix of head categories. Expect several Previous-Token heads (especially in early layers), some First/Positional heads, and a variety of other patterns.\n- **Output**: GPT-2 can produce reasonably coherent text for simple prompts. For factual prompts, it sometimes produces outdated or incorrect facts (it was trained on data from before 2019).\n\n## GPT-2 Variants\n\nThe dashboard supports all GPT-2 sizes, though only the small variant is in the default dropdown:\n\n- **GPT-2 Small** (124M params, 12 layers) -- in dropdown as \"GPT-2 (124M)\"\n- **GPT-2 Medium** (355M params, 24 layers) -- enter `gpt2-medium` in the dropdown\n- **GPT-2 Large** (774M params, 36 layers) -- enter `gpt2-large`\n- **GPT-2 XL** (1.5B params, 48 layers) -- enter `gpt2-xl`\n\nLarger variants have more layers and heads but use more memory and are slower.\n\n## HuggingFace Model IDs\n\n- `gpt2` or `openai-community/gpt2`\n- `gpt2-medium` or `openai-community/gpt2-medium`\n- `gpt2-large` or `openai-community/gpt2-large`\n- `gpt2-xl` or `openai-community/gpt2-xl`", "source_file": "gpt2_overview.md", "chunk_index": 1, "embedding": [-0.015519207, 0.010086174, 0.06606149, -0.02195496, 0.0065963184, -0.0032883286, -0.014549257, 0.048890747, 0.031352993, 0.029701456, 0.0056951824, -0.028574217, -0.02426187, -0.062863275, -0.046085756, -0.027604267, -0.019766018, 0.0014876943, -0.023226382, 0.025258034, -0.018966464, 0.0154929925, -0.006098236, 0.037172697, 0.010525273, 0.011554208, -0.020015059, 0.015571637, 0.036071673, 0.021732133, 0.01821934, -0.028207209, -0.034262847, -0.046033327, -0.0077923723, -0.012727323, 0.005413372, 0.054526947, -0.0012517604, 0.062496267, -0.0022921634, -0.0018743637, -0.018586349, 0.04170787, -0.030409258, 0.010276232, 0.030304398, -0.02021167, -0.074292965, 0.053452134, -0.031562712, 0.042494316, -0.04765865, -0.0060884054, -0.016594017, -0.00029245974, -0.03900774, -0.014968695, 0.027551837, -0.020119919, -0.006468521, 0.009450464, 0.037329987, 0.014116712, 0.035390086, -0.0017908039, -0.06527504, 0.059088334, -0.02278073, 0.005439587, 0.025900299, 0.014234679, -0.04144572, -0.039165027, 0.020696646, 0.022243325, -0.017105209, 0.032401588, 0.022413721, -0.0481043, 0.004417207, -0.036228962, -0.04535174, -0.032454018, -0.01899268, -0.0053511118, -0.012609356, -0.03481336, -0.040187407, -0.024353622, -0.008238025, -0.03106463, -0.036989193, -0.0020365682, 0.019792233, -0.009463571, 0.042127308, -0.014850728, -0.024576448, 0.036779474, 0.0068355296, 0.034524996, -0.030068465, 0.020552464, 0.0059704385, -0.03402691, -0.0068879593, 0.0056460295, -0.0473965, -0.019385902, -0.09730963, 0.021102976, -0.0089196125, 0.028180994, 0.0048956284, 0.012701108, -0.03127435, 0.05405508, 0.002698494, -0.0392961, -0.0034177648, -0.0040239836, 0.047842152, -0.006285017, 0.010191034, 0.01816691, -0.007844802, 6.220913e-05, -0.0035226243, 0.00057713693, -0.040056333, 0.038562085, 0.043988567, -0.024039043, -4.554323e-05, -0.0230822, -0.027106185, 0.02262344, -0.03449878, -0.026621208, 0.016698876, -0.030697621, 0.02410458, 0.068525694, -0.08525078, 0.01453615, -0.03360747, 0.044617724, -0.10790044, 0.064803176, -0.002198773, 0.02614934, 0.011370704, -0.023108415, -0.05214139, -0.048864532, 0.009516001, 0.023252597, 0.019058216, -0.04999177, 0.030016035, 0.00046859094, -0.003209684, 0.031955935, -0.02810235, 0.07560371, -0.013671059, -0.012884612, -0.0398204, -0.06443617, 0.039689325, 0.0055051246, -0.014260894, -0.030933555, 0.007353273, -0.0016253224, -0.08074182, 0.014575472, -0.044040997, -0.04338562, -0.013395802, -0.033240464, -0.009948546, 0.003922401, -0.055365823, -0.017013455, -0.01164596, 0.0025739733, 0.0053150663, 0.016607124, 0.025205605, 0.0125765875, 0.0747124, 0.023042878, -0.012779753, -0.05051607, 0.07182877, -0.015257059, 0.04663627, 0.02697511, 0.006632364, 0.036464896, 0.038064003, -0.038064003, 0.023449209, -0.016462943, 0.004135397, 0.0017006902, -0.027813986, -0.015938645, 0.005796765, -0.027735341, 0.010210695, 0.012819075, -0.07198606, 0.026057588, -0.0732968, -0.0069731576, 0.038929094, 0.055208534, 0.0010477758, 0.035599805, -1.2153821e-05, 0.0032719444, -0.02472063, 0.016174579, 0.038352367, -0.02563815, 0.0066192565, -0.007569546, 0.016279439, -0.008297008, -0.020447604, -0.009181761, -0.021260265, -0.008565712, 0.032768596, -0.022177787, -0.008310116, -0.062496267, 0.00018995546, 0.007680959, -0.008251132, 0.004177996, 0.046767343, -0.024484696, -0.029203374, -0.06375458, 0.033790976, 0.027735341, 0.031484067, -0.03536387, -0.043071043, 0.02789263, -0.010531827, 0.015361918, -0.026490133, 0.048078086, -0.037959144, -0.026110018, 0.006140835, 0.02482549, -0.031484067, 0.011698389, -0.022898696, -0.023711357, -0.034577426, 0.0032342605, -0.054579377, -0.056781426, 0.038221292, -0.0045417277, -0.0014246148, 0.038771804, -0.04231081, -0.014156034, 0.0017432894, 0.004931674, -0.037198912, 0.008355992, -0.0057607195, 0.0003782725, 0.046767343, 0.024078365, 0.030330613, -0.006828976, 0.019464547, -0.0041190125, -0.01693481, -0.020172348, 0.032322943, -0.035075508, 0.012976364, -0.034472566, -0.008297008, 0.020054381, 0.019084431, 0.019569406, -0.01092505, -0.0243143, 0.012884612, -0.03127435, -0.005482186, -0.034000695, -0.010649794, 0.030592762, 0.032506447, 0.06385944, -0.015178414, 0.0065078433, -0.038562085, -0.037749425, -0.003437426, -0.0054035415, 0.006422645, -0.0004890713, -0.0034865788, 0.02692268, 0.0018743637, -0.0278402, -0.0106104715, -0.030278184, -0.002968835, 0.027315903, 0.029229589, 0.037329987, -0.02088015, -0.015034232, 0.016751306, -0.020919472, 0.04084278, 0.01658091, -0.046741128, 0.041996233, -0.004377885, -0.0115804225, -0.000722138, 0.022885589, -0.023973506, 0.0056919055, -0.029858746, 0.005000488, -0.034629855, -0.0028606986, -0.050175276, 0.02912473, 0.024366729, 0.04181273, 0.033948265, -0.00893272, -0.0049873805, 0.00747124, -0.029570382, -0.0056919055, 0.028285854, 0.034524996, 0.018206233, -0.00497755, -0.020185456, -0.0045515583, -0.036989193, 0.02205982, 0.018822283, 0.014575472, -0.0036504217, 0.026791604, -0.021941852, 0.0033981036, 0.0043877154, -0.007130447, 0.0031605312, 0.015702711, 0.015728926, -0.026267307, -0.038404796, -0.0026427873, -0.0047285086, 0.045744963, -0.016279439, 0.024183225, 0.03229673, 0.029282019, 0.017419787, 0.0045646657, 0.0031818307, 0.04047577, 0.0096601825, 0.0050824094, -0.07292979, -0.003670083, 0.0012050652, 0.0026739175, -0.010387645, 0.007084571, -0.037225127, 0.012838736, 0.009345604, 0.0007114882, 0.024406051, 0.049572334, 0.0170921, 0.006347277, -0.016266331, -0.008939274, 0.020277208, 1.36194485e-05, 0.02369825, 0.033581257, 0.03987283, -0.015008017, 0.05594255, 0.012255455, 6.205553e-05, -0.020277208, 0.002477306, 0.005747612, 0.0102631245, 0.035154153, 0.012065398, 0.048943177, -0.024340514, -0.045115806, -0.059402913, -0.006068744, -0.055627972, -0.007779265, 0.024130795, 0.0007168131, -0.0038634175, -0.02359339, -0.011711497, 0.038299937, 0.045692533, -0.049965557, -0.008854075, -0.00497755, 0.0600845, 0.030251969, -0.00077579654, 0.0070649097, 0.024589555, 0.0051446697, 0.019556299, 0.0072353063, -0.019464547, -0.013310604, -0.038352367, -0.060556367, 0.004885798, -0.0322443, 0.01402496, 0.051145226, -0.028469358, 0.0074384715, -0.017105209, -0.023475423, 0.06831597, 0.012583141, 0.076232865, -0.034315277, 0.07895921, -0.0016310569, -0.025900299, -0.009529108, -0.014654117, 0.028652862, 0.0139201, 0.016148364, 0.0057607195, -0.017930977, 0.025376001, -0.05809217, -0.0016974134, -0.0266081, -0.00088475214, -0.009529108, 0.01069567, 0.013245067, -0.023226382, -0.029622812, 0.040868994, 0.012707662, 0.0017547584, 0.02420944, -0.016607124, 0.09306282, -0.03090734, -0.011914662, 0.038876664, 0.011429687, -0.049257755, -0.015125984, 0.025703687, -0.014667224, 0.03166757, 0.004492575, -0.018088266, -0.07088503, -0.03950582, 0.0028443143, 0.008867183, -0.00486286, -0.010997142, 0.047815938, -0.015597852, 0.01064324, 0.032322943, -0.001695775, -0.0029786655, -0.0018547026, -0.06165739, -0.01580757, 0.008179042, 0.02031653, 0.00029409816, 0.027551837, 0.037854284, 0.012688001, 0.009076902, 0.006671686, -0.0030982709, 0.0065012896, -0.05342592, -0.020067489, 0.0032801365, -0.028757721, -0.018953357, -0.045063376, -0.020290315, -0.008297008, 0.020631108, -0.03992526, 0.007890678, -0.02983253, -0.076128006, 0.031352993, 0.013959423, -0.023370564, -0.040030118, -0.0002967606, -0.0069338353, 0.023226382, 0.0099551, -0.0034079342, 0.00014151938, 0.027813986, 0.014903158, -0.0025182667, 0.0003721284, 0.05992721, 0.038876664, -0.016803736, 0.047763508, -0.016449835, -0.014614794, -0.014378861, -0.008290455, -0.0030802481, 0.006072021, -0.011986753, 0.031457853, -0.0031654465, 0.021705918, -0.01929415, -0.012733877, 0.03633382, 0.009076902, -0.023226382, 0.02993739, -0.032113224, -0.00775305, -0.0014082305, 0.012137488, -0.042127308, 0.007130447, 0.004014153, 0.0011485394, 0.020080596, 0.005462525, 0.029386878, -0.025952728, -0.012006414, 0.0007069825, 0.021574844, -0.016109042, 0.014378861, -0.00254612, -0.013526877, -0.032873455, 0.046190616, 0.04697706, -0.013959423, 0.014601687, 0.02655567, -0.020762183, -0.010544934, 0.03659597, -0.038273722, -0.05214139, -0.0009822387, -0.025874084, -0.025625043, -0.025625043, -0.00024043958, -0.024628878, 0.028416928, 0.0053314506, 0.022387506, -0.015243951, 0.0048726904, -0.017275605, -0.0037651118, -0.014011852, -0.029780101, -0.054474518, 0.031379208, -0.012996025, -0.014929373, 0.016764414, 0.007877571, 0.024445374, -0.00888029, -0.009103117, 0.00044565293, 0.015375026, 0.02123405, -0.011829464, -0.017603291, -0.004764554, -0.0034079342, -0.024379836, 0.00565586, 0.08577508, 0.010558042, -0.0138021335, 0.012917381, -0.006285017, 0.009424249, 0.032768596, -0.048943177, -0.016515372, -0.018075159, -0.011010249, -0.0202641, -0.012681447, 0.0059507773, -0.0109578185, 0.00013271281, -0.005777104, -0.0040075993, -0.024786167, -0.03268995, 0.029255804, 0.018638778, 0.007615422, -0.011036464, 0.01642362, -0.027604267, -0.03132678, 0.050725788, -0.061342813, 0.006848637, -0.025428431, -0.042153522, -0.016410513, 0.028941225, 0.01330405, 0.0070190337, -0.010492505, -0.017314928, -0.054789096, 0.032165654, 0.038037788, 0.0057902113, 0.015637174, -0.026214877, -0.0061604963, -0.05783002, -0.015322596, 0.016777521, -0.005088963, 0.036255177, -0.06385944, 0.023357457, 0.027630482, -0.013854563, 0.003152339, -0.009732273, 0.026345951, 0.0034472565, 0.00024248761, -0.038116433, 0.012366869, -1.4195459e-05, -0.040108763, 0.0025477584, -0.016344976, 0.052770548, -0.0017088824, -0.03767078, -0.03213944, 0.0038798018, -0.020080596, -0.0024838597, -0.014313323, 0.030278184, -0.015479885, 0.04524688, -0.0010895559, -0.024222547, -0.036176533, 0.013775919, -0.00990267, 0.009398034, -0.008664018, 0.0039617233, -0.01397253, 0.007005926, 0.007831695, -0.016961025, -0.033790976, 0.02496967, 0.0018481489, 0.031851076, 0.012550373, -0.017223176, -0.016279439, -0.025729902, -0.038562085, 0.015021125, 0.013618629, 0.035311442, -0.017419787, -0.006488182, -0.007569546, 0.0040239836, -0.010367984, -0.0053511118, -0.010086174, 0.00011069642, 0.008401868, 0.0019169629, -0.027394548, 0.0029245974, 0.015781356, -0.0010166457, 0.012137488, 0.0015679774, 0.010276232, -0.059350483, -0.0038011572, 0.0107022235, 0.007353273, 0.00035144322, 0.0051479465, 0.0023642543, 0.014968695, 0.0021430664, -0.0061277277, 0.049152896, 0.036517326, -0.0034931325, -0.02082772, 0.02067043, -0.01402496, -0.042127308, -0.0036012689, 0.008100397, -0.010073067, 0.015833786, -0.013422017, -0.010499058, 0.012773199, 0.020513142, 0.046295475, -0.060923375, 0.03696298, -0.00821181, 0.016082827, 0.00021279108, -0.00087410235, 0.005659137, 0.022938019, -0.0144968275, -0.0044040997, 0.008270794, -0.011016803, -0.003232622, -0.020604894, -0.015257059, -0.010171372, 0.008283901, -7.439496e-05, 0.0001865762, -6.61004e-05, 0.0070255874, 0.006776546, -0.112933695, -0.027158614, 0.013946315, -0.02896744, -0.0054789092, -0.003480025, 0.057463013, 0.01182291, 0.0066192565, -0.047108136, 0.033240464, -0.06663822, -0.003289967, 0.0034898557, 0.009168654, -0.04042334, -0.01499491, -0.011278952, 0.03213944, -0.019149968, -0.042127308, 0.017524647, 0.007897232, 0.02195496, -0.04999177, -0.003552116, 0.0026804712, -0.004390992, 0.0144968275, -0.011324828, -0.04702949, 0.046767343, 0.010367984, 0.02123405, -0.0073205046, 0.03127435, -0.010341769, -0.02666053, 0.03578331, -0.009987868, 0.0070190337, -0.029675242, 0.0037749424, -0.020709753, 0.030304398, -0.006776546, 0.01802273, 0.006193265, -0.011423133, 0.023108415, 0.0218501, 0.03166757, -0.004354947, 0.034289062, 0.015886215, -0.00047555426, 0.011482117, -0.035599805, 0.025035208, 0.03654354, 0.004400823, 0.025113853, -0.006822422, 0.005862302, 0.01555853, 0.0029442585, 0.03782807, 0.024812382, -0.016227009, -0.014221571, -0.028390713, 0.015021125, 0.0046564178, -0.018232448, -0.020971902, 0.010191034, 0.0144968275, 0.055470683, 0.009653629, -0.0020005228, 0.01540124, -0.036097888, -0.033423968, 0.010485951, 0.01883539, 0.022361292, 0.026581885, -0.025821654, 0.01842906, 0.02015924, 0.030592762, 0.00019620197, -0.0089654885, 0.02067043, -0.0262542, 0.0065701036, -0.012727323, 0.040554415, 0.039138813, 0.00030945844, 0.05803974, 0.007379488, -0.049257755, -0.000888029, -0.031903505, -0.0028246532, 0.01228167, -0.0036504217, -0.006255525, -0.022361292, 0.012819075, 0.036989193, 0.02681782, 0.014601687, -0.024524018, 0.025756117, 0.07764847, 0.0020136302, 0.0084346365, 0.004417207, 0.0045024054, 0.0028033536, 0.001959562, 0.018494597, -0.025651257, 0.022190895, 0.0054494175, 0.052010316, -0.0039813844, 0.011200307, -0.012766645, 0.036071673, -0.003350589, -0.003738897, 0.0065897647, 0.0022135188, 0.008408422, -0.021679703, -0.02072286, 0.00095274695, -0.029963605, -0.011652513, -0.0103548765, -0.02517939, 0.027289689, 0.002793523, -0.0080872895, -0.035259012, -0.0014516489, -0.00077006203, 0.02574301, -0.009686397, -0.015296381, 0.030881125, -0.0043156245, 0.0034931325, -0.020460712, -0.013789026, -0.041471936, 0.010597364, 0.025389109, 0.034734715, 0.016541587, 0.003289967, 0.038116433, -0.01340891, -0.0029557275, 0.008080736, -0.010027191, 0.016410513, -0.017930977, 0.046033327, 0.028285854, 0.02472063, 0.03407934, 0.030592762, -0.014208464, -0.01591243, 0.01918929, 0.029413093, 0.013671059, 0.032113224, 0.0027623929, 0.00849362, 0.0218501, -0.051040366, -0.01540124, -0.0025657811, -0.027289689, 0.034577426, -0.006465244, 0.005485463, 0.006170327, 0.0013992192, 0.011573869, -0.015112877, 0.03434149, -0.025467753, -0.020290315, 0.0044761905, 0.00949634, 0.012203026, -0.009188315, 0.0015393049, -0.01918929, 0.04514202, -0.0053347275, -0.0011362511, -0.0022987172, -0.006655302, -0.017328035, 0.02283316, -0.008664018, 0.005678798, -0.026647422, 0.016947918, 0.0071566617, 0.0092210835, -0.026490133, -0.004764554, -0.039636895, -0.0038011572, -0.03649111, -0.02302977, 0.011390365, 0.028023705, -0.005590323, 0.017419787, 0.010204141, 0.010846405, -0.0068879593, 0.0026509794, 0.0049644425, -0.006632364, 0.009974761, 0.001833403, 0.00944391, 0.021705918, 0.00752367, 0.0075498847, 0.04089521, 0.022243325, 0.0015925538, 0.0056230915, -0.00424681, -0.01453615, 0.013893886, -0.0048956284, 0.012379976, -0.010479397, 0.011600084, 0.012556926, 0.01261591, -0.03588817, 0.015243951, 0.01199986, 0.023986613, 0.03777564, 0.0052134837, -0.0151915215, 0.001974308, 0.045535244, 0.022308862, -0.014785191, 0.031169489, -0.00049767306, -0.009810918, -0.025349787, 0.040868994, 0.009463571, -0.031536497, 0.034367707, -0.024458481, -0.005482186, 0.010400753, 0.0034996863, 0.0141953565, 0.014732761, 0.023632713, -0.0070452485, -0.0329521, 0.010518719, 0.00047924073, -0.001097748, 0.014300216, -0.018114481, -0.009994422, -0.021365125, 0.028128564, -0.011986753, 0.015309488, 0.012058844, 0.019412117, -0.005773827, -0.0022839713, 0.019254828, 0.0052429754, -0.013140207, 0.016095934, -0.022400614, -0.001097748, -0.020591786, 0.023396779, -0.05305891, 0.0026903017, 0.033135604, 0.036517326, -0.0576203, 0.002511713, 0.020565571, -0.016646447, -0.015099769, 0.01642362, 0.02123405, 0.026359059, 0.010571149, 0.0013959423, -0.025166282, -0.0076023145, 7.4241354e-05, -0.00059515965, 0.009817472, 0.014509935, -0.007838248, 0.01791787, -0.030278184, 0.012170257, -0.006465244, -0.013815241, -0.0058491947, 0.04508959, 0.007910339, 0.013684167, 0.029622812, -0.05992721, -0.032454018, -0.031221919, -0.016161472, 0.0056460295, -0.027368333, 0.019464547, 0.008270794, -0.019634943, -0.011324828, 0.04165544, -0.0024805828, 0.014654117, -0.0243143, 0.02399972, -0.00053822424, 0.048812103, -0.022204002, -0.0278402, -0.0015532315, 0.0032571985, 0.04396235, 0.01007962, -0.00798243, -0.028364498, -0.010433521, 0.022938019, 0.017000347, -0.012189918, 0.012203026, 0.0199102, 0.0088278605, -0.020434497, -0.0047547235, 0.038195077, -9.262249e-05, 0.0051938226, 0.029150944, -0.010636686, -0.008768877, 0.021732133, 0.03263752, 0.03481336, -0.01499491, 0.018009622, 0.006163773, -0.018494597, -0.054422088, 0.0050561945, 0.030199539, -0.011423133, 0.016554695, 0.000998623, 0.025507076, -0.046400335, -0.0051512234, 0.0026903017, -0.0055116783, 0.016843058, -0.020329637, -0.010059959, -0.008906505, 0.0047547235, -0.014903158, 0.011180646, -0.013422017, 0.0054920167, 0.012445513, -0.021404447, -0.007150108, 0.030592762, -0.030383043, -0.00013568248, 0.034105554, 0.006189988, 0.010433521, -0.050384995, 0.024877919, -0.017629506, 0.018664993, -0.0008245398, -0.048549954, -0.029098514, -0.005537893, -0.0024871365, 0.024419159, -0.020644216, -0.0044040997, -0.023069093, -0.029413093, 0.032532662, -0.0031703617, -0.0170921, 0.0042631943, 0.0007901328, 0.009457017, 0.004390992, -0.002134874, -0.006216203, 0.0070518022, -0.01346134, -0.022295754, 0.042546745, -0.017603291, -0.022938019, 0.029413093, 0.0103090005, 0.010518719, 0.012937042, 0.04702949, 0.0031425084, 0.008126612, 0.0027722234, -0.014326431, 0.0034079342, 0.020224778, 0.011429687, -0.0057148435, -0.024904134, 0.0088737365, -0.0060425294, -0.0044368682, -0.014680332, 0.0062653557, 0.024039043, 0.00588524, 0.008414975, -0.0052691903, 0.01975291, 0.010577703, 0.017183853, 0.016358083, 0.0135006625, 0.051197655, -0.011704943, -0.04498473, 0.038352367, -0.020683538, 0.009103117, -0.007988984, 0.0017367357, 0.0071435543, 0.004836645, -0.009745381, -0.04917911, -0.014772084, 0.0024822212, 0.036464896, 0.022282647, -0.014562365, 0.00944391, 0.031169489, 0.0031343163, -0.024458481, -0.029098514, 0.01740668, 0.004204211, 0.03127435, -0.009535662, -0.02123405, 0.011088894, -0.0076481905, 0.0021905808, -0.015099769, -0.013710381, -0.01601729, -0.02912473, -0.04068549, 0.0052200374, 0.0074384715, -0.005442864, -0.045928467, -0.0021512585, -0.0004718678, -0.0055280626, -0.008886844, -0.0111413235, 0.0012861674, -0.014300216, -0.008958935, 0.0024248762, 0.015977968, 0.01136415, 0.011475563, -0.0072025377, 0.009404588, 0.032506447, -0.012838736, -0.0017269051, 0.030409258, -0.0026886633, -0.017524647, 0.005485463, 0.024419159, -0.0115804225, 0.032925885, -0.0035652234, 0.022924911, -0.03014711, -0.029098514, 0.0044532525, 0.01735425, 0.004882521, 0.005606707, -0.009509447, 0.008224918, -0.0036143763, -0.026948895, -0.01652848, -0.008696786, 0.0015008018, 0.04917911, 0.015414348, 0.020382067, 0.038011573, -0.032873455, -0.0086181415, -0.0179703, -0.023069093, -0.013218852, -0.051695738, 0.011573869, 0.006534058, -0.0066684093, 0.018769853, 0.007058356, 0.010046852, -0.0027755003, -0.041996233, 0.026634315, 0.03599303, -0.028521787, 0.0036635292, -0.017262498, 0.006946943, -0.02517939, 0.023619605, -0.008886844, 0.046819773, 0.0032571985, 0.00012390625, -0.012871505, 0.010787422, -0.018664993, 0.029360663, 0.0072746286, 0.007589207, -0.044932302, 0.022662763, -0.0016810291, -0.0144968275, -0.008972042, -0.017734366, 0.008310116, -0.00081593805, 0.0077596037, -0.016711984, 0.03090734, -0.017026562, 0.02088015, -0.0062030954, 0.035809524, 0.0059409468, 0.010649794, -0.048549954, 0.018573241, 0.018547026, 0.02021167, -0.020709753, -0.036307607, -0.03557359, -0.016685769, -0.03578331, 0.00413212, 0.012812521, 0.009751935, -0.040134978, 0.00022405529, 0.011069233, -0.0028180995, 0.010269678, 0.029229589, 0.003853587, -0.03397448, -0.021928744, 0.008893398, -0.013775919, -0.0042566406, -0.012976364, -0.0015499546, 0.00067830994, 0.0366484, 0.007890678, 0.015427455, -0.0015466778, -0.0008503451, 0.013946315, -0.018900927, -0.010204141, -0.01453615, 0.027787771, 0.01873053, 0.014837621, 0.0038404795, 0.010682562, 0.0028688908, -0.027394548, 0.009869901, 0.011462456, -0.017118316, 0.016594017, 0.0310122, -0.0060130376, -0.04283511, -0.03654354, 0.056886286, -0.016213901, 0.0083887605, 0.022348184, -0.009732273, 0.009994422, 0.0072025377, -0.004276302, 0.032349158, -0.008526389, -0.0024035766, 0.020840827, -0.03961068, -0.022531688, 0.0022888866, 0.002534651, -0.0027869693, -0.002074252, 0.023855539, 0.018415952, 0.02655567, 0.0043844385, 0.02329192, 0.021089869, -0.021614166, 0.050280135, -0.009004811, -0.0046334798, 0.03654354, 0.027001325, -0.034996863, 0.03386962, 0.008670571, 0.04283511, 0.009876455, 0.00775305, -0.007405703, -0.016449835, 0.012642125, 0.05051607, 0.002877083, -0.028993655, -0.028783936, -0.018560134, -0.027813986, -0.03187729, 0.044722583, -0.011783588, -0.064960465, -0.0070452485, -0.0070714634, 0.010479397, 0.01580757, -0.0023560622, -0.00844119, 0.0118818935, 0.016738199, -0.012583141, 0.0013541622, -0.016567802, -0.02456334, -0.023527853, -0.010426967, 0.019005787, 0.005583769, 0.0057836575, 0.030068465, -0.007877571, -0.039846614, -0.0044073765, -0.007798926, -0.016187686, 0.016253224, 0.029150944, -0.0079955375, 0.009148993, 0.01261591, -0.03696298, -0.017537754, 0.029282019, 0.013697274, 0.024851704, -0.03837858, -0.022911804, -0.014509935, -0.00069919997, -0.0074384715, -0.0065766573, 0.021836992, 0.0062981243], "content_hash": "bc25da33ee2624f1ba17cda0440ce0bd"}, {"content": "# Attention Head Categories Explained\n\n## What Are Head Categories?\n\nThe dashboard automatically analyzes all attention heads in the model and categorizes them based on their behavior patterns. This helps you understand what each head is doing without having to inspect every attention map manually.\n\nHead categories appear in **Stage 3 (Attention)** of the pipeline. Click any category to expand it and see which specific heads (like L0-H3, L2-H11) belong to it.\n\n## The Five Categories\n\n### Previous-Token Heads\n\n**What they do**: These heads strongly attend to the **immediately preceding token**. For every token at position *i*, the head focuses most of its attention on position *i-1*.\n\n**Why they matter**: Previous-token heads help the model track local context -- the word that just came before. They're important for bigram patterns (common two-word combinations like \"of the\" or \"in a\").\n\n**Detection**: A head is classified as Previous-Token if, on average, more than 40% of each token's attention goes to the token directly before it.\n\n**In the dashboard**: These heads are labeled with a purple color. Ablating them often causes noticeable changes in predictions.\n\n### First/Positional Heads\n\n**What they do**: These heads focus heavily on the **first token** in the sequence or show strong **positional patterns** (always attending to a specific position regardless of content).\n\n**Why they matter**: The first token often serves as a \"default\" attention target. Positional heads help the model keep track of where it is in the sequence.\n\n**Detection**: Classified when average attention to the first token exceeds 25%.\n\n### Bag-of-Words (BoW) Heads\n\n**What they do**: These heads spread their attention **broadly and evenly** across many tokens, without focusing strongly on any particular one.\n\n**Why they matter**: BoW heads capture a general summary of the entire input. They help the model maintain an overall sense of what the text is about.", "source_file": "head_categories_explained.md", "chunk_index": 0, "embedding": [0.009047568, 0.026902948, 0.008252593, 0.023672575, -0.015975203, -0.050398864, -0.014082407, 0.009091733, -0.013123389, 0.032354202, 0.015760686, -0.0024101608, -0.016013058, -0.00077052595, 0.0041073686, 0.01703517, -0.00977314, 0.008019148, 0.013628135, 0.056581996, -0.0036594067, 0.014095025, 0.040707745, 0.059408575, 0.030713778, -0.0048234765, -0.02410161, 0.041893896, 0.0030805264, 0.013628135, 0.007609042, -0.009697428, -0.04151534, 0.005407089, -0.016808033, 0.008946618, -0.020114118, 0.00976052, -0.009659572, 0.019672465, 0.0058739786, 0.007337741, -0.044619523, 0.00688347, -0.048127506, 0.007918199, 0.042070556, -0.019962694, -0.026928185, 0.06657597, -0.026246779, 0.032783236, -0.018284414, 0.008296758, 0.03439842, 0.0062304554, -0.02753388, 0.027180558, 0.01740111, -0.01684589, 2.5908885e-06, -0.02445493, 0.011615462, 0.010315741, 0.025313, -0.051786914, -0.05183739, -0.036871675, 0.0025521207, -0.0026278326, 0.069503486, 0.0400011, 0.010385144, 0.0073566693, 0.021186702, -0.022271905, -0.009596478, 0.016038297, -0.03078949, -0.0013170709, 0.0022555825, 0.033944152, 0.0102652665, -0.04159105, -0.011558677, -0.051130746, -0.023230923, -0.050676472, -0.018953202, -0.062739894, -0.017603008, 0.025325617, -0.018107753, -0.0015008299, -0.04835464, -0.039395407, 0.01859988, -0.034247, 5.5502314e-05, -0.006624788, 0.034549847, -0.025943931, -0.009520766, -0.016404238, 0.030183796, -0.0487332, 0.012145445, -0.019243432, -0.024921821, -0.028871456, -0.054361116, -0.0049338895, -0.008353542, 0.038764473, 0.048859388, 0.0039843367, -0.020353872, 0.04585615, 0.018991059, -0.020013168, 0.012631263, -0.032328963, 0.028492898, 0.0031720116, -0.006984419, 0.0070853685, 0.03078949, -0.015457839, -0.0025994405, -0.025464423, 0.026726287, 0.010479784, 0.017451584, -0.03813354, -0.013224338, -0.017666101, -0.0026041726, 0.02232238, -0.06758545, -0.004356587, 0.021464312, -0.01330005, 0.013249576, 0.011205356, 0.004268256, 0.0036594067, -0.0091295885, 0.07540902, -0.017779669, 0.027836727, -0.015180228, -0.00035765968, -0.032051355, -0.04075822, -0.022423329, -0.014738576, 0.025085863, 0.012505076, -0.0015284332, -0.050398864, 0.021249795, 0.0206441, 0.023407584, 0.0062935483, -0.051307406, -0.0042020082, -0.033515118, -0.009949801, -0.018915346, -0.022385474, -0.019180339, 0.0127574485, -0.008441873, -0.021110991, 0.019647228, 0.019357, -0.038764473, 0.015104516, -0.00047753679, -0.02753388, 0.01953366, 0.029149067, -0.024631592, -0.015886873, -0.030562354, -0.010549186, 0.016669229, 0.015697593, -0.040606797, 0.021981677, -0.01814561, 0.0050537665, 0.04772371, 0.05163549, 0.003962254, -0.03285895, 0.022650465, 0.020051025, 0.027306745, -0.01589949, 0.021413838, -0.0124672195, 0.037982117, 0.0055774404, 0.00093062496, 0.052342135, -0.0010576, 0.035382677, -0.046815168, -0.047395624, 0.004119987, -0.06758545, 0.032404676, 0.042827677, -0.08469634, -0.0067951395, -0.067131184, -0.0022666238, 0.018953202, 0.021237178, 0.060216166, 0.005123169, 0.049717456, 0.04719373, -0.06390081, 0.0009661149, 0.05683437, -0.014536678, -0.01153975, -0.022751415, 0.020707194, 0.02547704, 0.009501839, -0.0050726947, -0.038310204, -0.06359796, 0.031294238, -0.040329184, 0.055421084, -0.03220278, -0.020177212, 0.02315521, -0.002301325, 0.013804796, 0.018940585, 0.005318758, 0.004996983, -0.036619302, 0.051786914, 0.029098593, 0.09302464, 0.01581116, -0.0806079, 0.023849236, -0.035862185, 0.01581116, 0.0023943877, 0.038461626, -0.025085863, 0.034373187, 0.00083992846, 0.009615406, -0.008927691, -0.009785758, -0.015558788, -0.053654473, -0.02559061, -0.003962254, -0.050752185, -0.014561915, -0.06001427, -0.0052241185, 0.033565592, -0.002195644, -0.035155542, 0.004949663, -0.02902288, -0.01395622, -0.027029134, -0.024883965, 0.01823394, -0.005120015, 0.060771387, 0.030890439, -0.02781149, 0.0032398368, -0.011867834, -0.028366711, 0.00258209, -0.00516418, 0.0065112202, -0.026398202, 0.04974269, -0.018536787, -0.012044495, 0.013085534, 0.014435728, 0.008189499, -0.011318923, -0.0018218167, 0.027483406, -0.054462064, 0.02121194, 0.017514678, 0.034171287, -0.016177101, -0.0018407446, -0.018675592, -0.012265322, -0.032884184, -0.02922478, -0.032909423, 0.027836727, -0.007552258, -0.04903605, -0.038083065, 0.025363473, 0.008454491, -0.0032019808, 0.008927691, -0.04875844, -0.051080268, 0.014814287, 0.017994186, 0.038966373, -0.035155542, -0.005403934, 0.019180339, -0.030360457, -0.053452574, 0.049515557, 0.007848796, -0.021035278, -0.005170489, -0.04063203, -0.014170737, 0.016227575, 0.022814509, -0.048607014, 0.06400176, -0.03268229, -0.01693422, -0.042020082, 0.0035016737, -0.03972349, 0.059408575, -0.017375872, -0.01395622, -0.012012948, -0.039143033, -0.041742474, -0.0021451693, 0.011236902, -0.040909644, -0.042373404, 0.06682833, -0.02530038, -0.019735558, -0.005918144, 0.004643661, -0.0006541978, -0.026549626, 0.008208428, 0.016076151, 0.030562354, -0.031142812, -0.025426567, -0.033060845, 0.0053818515, 0.01088989, 0.015306415, -0.021123609, -0.016883746, -0.06324464, -0.060165692, -0.010542877, -0.01024003, -0.013665991, -0.019407474, 0.047597524, -0.017779669, -0.0055427393, -0.0355341, -0.01833489, 0.059963793, 0.02912383, -0.02912383, 0.002897556, -0.028240524, -0.027382456, -0.04810227, 0.006687881, -0.01144511, 0.015268559, -0.031899933, 0.026019644, -0.002802916, 0.046461847, 0.026044881, 0.02427827, 0.019773414, -0.024795635, -0.039395407, -0.0184106, 0.038158778, -0.0062020635, -0.0028597, 0.010662754, 0.014082407, -0.0054733367, 0.06733308, 0.028619085, 0.023874473, 0.0040253475, -0.030032372, 0.04045537, -0.0070348936, 0.01730016, 0.0024716768, 0.03210183, -0.0031988262, -0.05572393, -0.0532002, 0.018297033, -0.06738356, -0.017502058, 0.044064302, -0.06364844, 0.031067101, -0.04719373, -0.0055080377, 0.075964235, 0.028619085, -0.018902728, 0.0053913156, -0.0370231, 0.015483076, 0.03230373, 0.0032776927, -0.015419982, -0.0004716218, -0.039673015, -0.012265322, -0.016770177, -0.010681682, 0.012063423, -0.022675702, -0.0770242, 0.028619085, -0.039319694, 0.002567894, 0.048051793, 0.0041452246, 0.03598837, -0.047244202, -0.015520931, 0.028972406, 0.008492347, 0.025830364, -0.023571625, -0.01833489, 0.021287652, -0.020051025, 0.013994075, -0.016808033, -0.011419873, -0.009924564, -0.037098814, 0.0534021, -0.017615626, 0.033717014, 0.018625118, 0.017741812, -0.049919356, 0.014536678, -0.032228015, 0.010391453, -0.027432932, -0.0036436333, -0.06011522, 0.023129974, -0.017539915, 0.002219304, 0.009407199, 0.043837167, 0.034347948, -0.011318923, 0.006574313, 0.022663085, -0.002238232, -0.026272016, -0.015798543, 0.026120592, -0.012858398, 0.0069654915, 0.0059212986, -0.04532617, -0.015861634, 0.0061137327, -0.008303068, 0.05330115, -0.038360678, 0.03934493, 0.045376644, 0.0011695905, 0.012637571, 0.03197564, 0.0076847537, -0.031067101, 0.039143033, -0.035685524, 0.0037792837, 0.0008422945, 0.03227849, -0.008833051, -0.0028928241, 0.012587097, 0.027508643, -0.012675428, -0.0035742307, -0.005691008, 0.0025363474, 0.001795002, -0.023495914, 0.0010489248, -0.00039610712, -0.013628135, -0.051484067, -0.031672794, -0.006176826, -0.0052146544, -0.035180777, 0.00014274841, 0.0044259895, -0.030385694, 0.0023218305, -0.0005903159, -0.05304878, -0.03230373, 0.03730071, 0.027735779, -0.010669063, 0.03369178, 0.0025189968, 0.029149067, -0.007981292, 0.023950186, -0.0153190335, 0.023294017, 0.058348607, -0.03452461, -0.04419049, 0.03376749, -0.018372744, -0.031698033, -0.03051188, -0.031395186, -0.0074008345, -0.0039496357, -0.018940585, 0.03248039, -0.03210183, 0.006921326, -0.019861745, -0.025439186, 0.031521372, -0.004435453, -0.0018091981, 0.011949856, -0.053654473, 0.008498657, -0.026852474, 0.02912383, -0.04121249, 0.027054371, 0.01395622, -0.0130224405, 0.017703958, 0.02056839, 0.016214957, -0.034373187, -0.015621881, -0.00553643, -0.014801669, -0.0027272042, -0.009735283, 0.030183796, -0.00019667338, -0.017375872, -0.005202036, 0.03407034, -0.03263181, 0.0028597, 0.049666982, -0.03719976, -0.0074386904, 0.016290668, 0.017312778, -0.012719593, 0.03210183, 0.013590279, 0.016543042, -0.0013754321, 0.024972295, -0.021047898, -0.0041420697, -0.03916827, -0.0023423356, 0.018360127, 0.021174084, -0.013047677, -0.0034543537, -0.050020304, -0.03134471, -0.019016296, 0.03023427, -0.018650355, -0.017363254, -0.018991059, -0.0075396397, 0.008700555, -0.0051137055, 0.006902398, -0.0035174468, -0.0017965793, 0.032909423, 0.025615847, -0.043382898, 0.015659736, 0.0012871017, -0.036568828, -0.016265431, 0.02037911, -0.009810995, -0.0295781, 0.0088393595, -0.015571406, 0.003470127, 0.036392167, -0.004072667, 0.00011425788, 0.019735558, -0.020290779, -0.010170626, -0.006574313, -0.00067904074, 0.0112558305, -0.010479784, -0.0340451, -0.020631483, 0.020240305, -0.04048061, 0.068695895, 0.0060632583, -0.0006009629, -0.0171235, -0.01404455, 0.016719703, -0.0030347838, 0.01740111, -0.037553083, 0.009716355, -0.012069733, -0.045906626, -0.015079279, 0.0008635884, 0.0060601034, 0.005684699, 0.0088141225, -0.009602788, -0.085705824, 0.02819005, 0.04355956, 0.024959678, 0.029552864, 0.0075396397, 0.010574424, -0.0053881607, -0.03182422, 0.022663085, -0.022486422, 0.02846766, -0.029679049, -0.041262966, 0.015937347, 0.0022634692, 0.028593846, -0.029779999, 0.015394745, -0.00059386494, 0.012788995, -0.034196526, 0.012871017, 0.000911697, 0.008012839, -0.005088468, -0.0027950297, 0.033338457, -0.015180228, -0.052443083, -0.021729304, -0.006732046, -0.029325727, 0.00688347, -0.00019203998, -0.003943326, 0.027458169, 0.07662041, 0.0023170984, -0.023003787, -0.038385913, 0.011905691, 0.008826741, 0.043281946, -0.0057919575, 0.030688541, -0.033742253, -0.016038297, 0.0039780275, -0.002075767, -0.04027871, 0.025641084, -0.004921271, 0.0022035306, 0.021540025, -0.0064418176, -0.020543152, -0.03962254, -0.02781149, 0.010599661, 0.023268778, 0.033616066, -0.01916772, -0.046588033, -0.007703682, 0.042121034, -0.056581996, 0.015104516, -0.010378835, -0.009697428, 0.032354202, -0.024581118, 0.0039717183, -0.007419762, 0.033616066, 0.002400697, 0.0011270026, -0.004359741, -0.02567894, -0.03422176, -0.007508093, 0.009893017, -0.007899271, -0.029880948, 0.01981127, -0.009060186, 0.006176826, -0.016366381, 0.021502169, 0.033918913, 0.0059907013, 0.008208428, -0.020694576, 0.004631042, -0.02027816, -0.033338457, -0.02074505, 0.008612224, -0.002697235, -0.0141833555, 0.0023139438, 0.00544179, 0.032228015, -0.028164813, 0.02874527, -0.032328963, 0.05274593, -0.014448347, -0.03439842, 0.0071673896, 0.0071989363, -0.024076372, -0.022259288, -0.010467165, -0.008208428, 0.0030663304, -0.03507983, -0.030259507, 0.00362786, -0.03770451, -0.0051578707, -0.010208483, 0.018196084, -0.03581171, -0.030259507, -0.01943271, -0.017893236, -0.085201085, -0.017325398, 0.029906185, -0.022221431, -0.03432271, 0.0007579073, 0.040581558, 0.030183796, 0.023394965, -0.015836397, -0.0027745243, 0.0029369893, -0.00046767847, -0.018069897, 0.028366711, -0.036594067, 0.00049409876, 0.0122968685, -0.0032303727, 0.0007835389, -0.03747737, 0.009463983, 0.013287432, -3.206417e-05, -0.03116805, 0.009987656, -0.028997643, -0.013224338, 0.03889066, 0.031698033, -0.017792288, 0.01954628, -0.03487793, 0.015495694, 0.0060790316, 0.03974873, -0.002563162, 0.01748944, 0.022827126, -0.028391948, 0.009697428, -0.017577771, 0.007331432, -0.02037911, 0.0071106055, -0.0006589298, -0.004511165, -0.007861415, -0.0020789215, 0.037073575, -0.0028770508, 0.004261947, 0.020101499, 0.029325727, 0.037906405, -0.012113898, -0.035130303, -0.01591211, 0.0228776, 0.022814509, -0.011634389, -0.0062777754, 0.027937677, -0.007779394, 0.0051831077, -0.00790558, -0.0007547526, 0.026953423, -0.039799202, -0.022827126, -0.019117245, -0.017249687, 0.00753333, 0.008978165, 0.0046657436, -0.009457673, 0.011419873, 0.00014136825, 0.017464202, 0.0020741895, 0.071572945, -0.014082407, -0.034272235, 0.016114008, 0.027508643, 0.026095355, -0.0010158009, -0.012473529, 0.016265431, 0.02074505, -0.014435728, -0.03285895, 0.011117025, 0.035357438, -0.03389368, -0.007836178, 0.033136558, 0.020883854, 0.06682833, -0.0019527351, 0.027710542, 0.006776212, 0.011318923, -0.036997862, -0.05004554, -0.027735779, -0.013199101, 0.0017918474, -0.022208814, -0.030259507, 0.031092338, 0.019382237, 0.017350635, 0.028240524, -0.018612498, -0.003489055, 0.031193286, 0.03116805, 0.009911944, -0.011274758, -0.021464312, 0.025754651, -0.0068897796, 0.024593737, -0.00012766519, -0.00063290383, -0.028265761, -0.0048234765, 0.016341144, -0.018662974, 0.007867725, 0.008605915, -0.0031483516, -0.0011822091, 0.014334779, -0.011123334, 0.008965546, 0.018397983, -0.01338838, 0.0057099364, -0.019584134, -0.010057059, 0.06874637, 0.011968783, 0.0010639094, 0.026852474, -0.019281287, -0.04706754, -0.0105807325, 0.04027871, -0.0045269383, 0.012391508, -0.011956165, 0.015558788, -0.0036530974, 0.022170957, 0.016972076, -0.026448676, 0.0020063643, 0.04855654, 0.003246146, 0.0055017285, 0.06960444, -0.00062225683, 0.007545949, 0.018852254, -0.02801339, 0.024215177, 0.0046278876, 0.018297033, -0.028164813, -0.0042998027, 0.044341914, 0.04492237, -0.0075270208, 0.055875354, -0.013262195, -0.004444917, 0.012511385, -0.0028754734, -0.019243432, 0.017274924, -0.013173864, 0.005675235, 0.0043029576, 0.0442662, -0.013009822, 0.0062178364, -0.012801614, -0.04025347, 0.036568828, 0.03268229, -0.0184106, 0.022751415, -0.03712405, -0.03712405, 0.026726287, 0.014751194, -0.017703958, 0.017186593, -0.008978165, 0.023306634, -0.002157788, -0.0045900317, 0.0021104682, 0.036038846, -0.013085534, -0.009911944, 0.00958386, 0.009640643, -0.0024842955, 0.03200088, -0.03581171, 0.010896198, -0.038158778, -0.002959072, 0.04772371, -0.012265322, 0.039673015, -0.0026562244, -0.03591266, -0.017653482, -0.03359083, 0.013337906, 0.013943601, 0.026675813, 0.043988593, 0.014637627, 0.00827783, -0.015886873, -0.029552864, -0.01330005, 0.018347507, 0.013375762, -0.01088358, -0.010934055, 0.016240194, -0.013426237, 0.006299858, 0.002299748, 0.025439186, 0.023003787, -0.025628464, -0.0026073272, -0.001067064, -0.0095649315, 0.016454712, -0.0049054977, 0.006009629, -0.012347342, -0.005807731, 0.0071863174, 0.028316237, -0.028694795, -0.0038833877, -0.002758751, 0.023609482, 0.014372635, 0.034852695, 0.008599605, 0.03359083, 0.029048117, -0.008713174, -0.04063203, -0.010359907, 0.0069276355, -0.02632249, 0.019584134, -0.00183759, 0.0014069788, -0.014864762, 0.03197564, -0.011495585, 0.0086942455, 0.017085643, 0.01218961, 0.008025457, -0.03061283, 0.014322161, -0.020972185, 0.031319473, -0.014107644, 0.0044953916, -0.009331487, 0.016164482, 0.015079279, -0.027559118, -0.01953366, 0.03179898, -0.04270149, -0.011735339, -0.023142593, -1.2495415e-05, -0.0027934522, -0.022574753, -0.0020915403, 0.0063850335, 0.0075585674, 0.0138678895, -0.028290998, 0.013943601, -0.011994021, 0.03338893, -0.0033187033, -0.005523811, -0.0051105507, 0.016025677, -0.061528508, -0.025729414, 0.006921326, 0.011369398, 0.0064986013, -0.015558788, 0.05355352, -0.01684589, -0.025830364, 0.013047677, -0.015672356, 0.014372635, -0.0629418, 0.0065112202, 0.040404897, -0.033994626, 0.0045427117, 0.01600044, -0.022170957, 0.011489275, -0.0147638135, -0.020681957, -0.0054260166, 0.0036530974, 0.00014728324, 0.020076262, 0.019369617, -0.0412882, 0.0127574485, -0.013148626, 0.0015899491, 0.02344544, -0.017691338, -0.006038021, 0.026751524, -0.0038549956, 0.015407364, 0.0442662, 0.0015252786, 0.006586932, -0.004517474, -0.010158008, 0.009135898, 0.06743403, -0.014587152, -0.037754983, 0.016959457, -0.005356614, 0.009167445, 3.9236096e-05, -0.019508423, -0.028316237, -0.03013332, -0.021249795, 0.019798651, -0.016984694, 0.017653482, -0.0059118345, -0.020303398, -0.0050726947, 0.014625008, 0.03972349, -0.0022634692, 0.01822132, 0.01730016, -0.049086522, -0.010088606, 0.018549407, 0.012858398, 0.009596478, 0.0018218167, -0.013085534, -0.011363088, -0.015949966, -0.03674549, -0.01470072, -0.009697428, 0.0017413729, 0.02753388, 0.0046531246, 0.010006584, -0.0076973727, -0.0023360264, 0.0015244898, 0.02074505, -0.0005982026, -0.0055585126, -0.013640754, 0.018347507, 0.02408899, 0.022524279, -0.016391618, -0.038587812, -0.020139355, 0.007394525, 0.0009866202, -0.010687991, 0.022612609, -0.024732541, -0.006504911, 0.023634719, 0.0075270208, 0.028947169, 0.00061594753, 0.037628796, -0.02512372, -0.016871126, -0.013375762, -0.03230373, 0.0020300243, -0.019912219, -0.0029653811, 0.021275032, -0.016517805, -0.036568828, 0.004391288, 0.008366161, 0.037603557, -0.01814561, -0.01506666, 0.01275114, -0.008120097, 0.006444972, -0.0061799805, -0.02930049, -0.04373622, 0.004448072, -0.018839635, -0.0069150166, 0.017830143, -0.029477151, -0.032985136, 0.032884184, 0.0059970105, 0.02176716, 0.02874527, 0.047118016, 0.0086690085, -0.007369288, -0.010460855, -0.037603557, 0.007836178, -0.023647338, 0.004855023, -0.024126846, -0.02884622, 0.028694795, -0.030562354, -0.007552258, -0.04272673, 0.0075396397, -0.004829786, -0.011382016, -0.0052966755, -0.010189555, -0.0020678802, 0.025830364, -0.008549131, 0.0036436333, -0.017653482, 0.039496355, -0.024303507, -0.009943491, 0.036997862, -0.008422945, 0.007836178, 0.007937127, -0.0026672657, 0.0059780823, 0.01963461, 0.018574644, -0.019874364, -0.01703517, -0.027155321, 0.021363365, 0.0013241689, -0.021249795, -0.015394745, 0.0064512817, 0.019369617, 0.0056973174, -0.032581337, 0.027155321, 0.028770508, -0.0035395296, -0.01571021, -0.009274703, 0.024669448, 0.021224558, 0.012252703, 0.019558897, -0.009634335, 0.0017886928, 0.005397625, -0.011577605, 0.028038627, 0.01330005, -0.0010228988, -0.032581337, 0.013363143, -0.0064986013, -0.012921491, -0.02455588, 0.0094198175, -0.0035963135, -0.025615847, -0.0053881607, 0.007867725, 0.051786914, -0.014915236, 0.0073566693, 0.030360457, 0.019420093, 0.018473694, -0.027281508, -0.03730071, 0.019331763, -0.009476601, -0.010523949, -0.0049717454, 0.016290668, 0.010088606, 0.018107753, -0.037578322, 0.012044495, -0.008151644, -0.031142812, 0.017880619, 0.010852033, -0.019205576, -0.006485983, 0.009243157, 0.024391837, -0.0010023935, -0.037149288, -0.019281287, -0.010227411, 0.023432821, 0.03674549, 0.0028581228, 0.015962584, 0.035761237, -0.018827016, 0.020593626, -0.014309542, -0.025426567, -0.008183191, -0.015407364, 0.026852474, 0.002219304, -0.0071232244, 0.017375872, 0.013287432, -0.028063864, -0.010334669, -0.03795688, 0.0027145857, 0.01200033, 0.022246668, 0.01013908, 0.0049401987, -0.008984474, -0.041162014, -0.0007027007, -0.0059591546, 0.06157898, -0.0065806224, -0.017136117, 0.03598837, -0.0355341, -0.018625118, -0.008612224, -0.043837167, 0.023218304, -0.0708663, -0.00079379155, 0.020997424, 0.025603227, 0.011489275, -0.020025788, 0.0243666, -0.00553643, 0.0038928515, 0.018372744, 0.040985353, -0.035155542, 0.012221157, 0.007861415, 0.02677676, -0.0027240496, -0.010006584, -0.01376694, -0.00087857305, -0.004429144, 0.023432821, -0.03608932, -0.04835464, -0.010101224, -0.034247, 0.015886873, 0.0052714385, 0.009975038, -0.00087857305, -0.0676864, 0.008082241, -0.0052304277, 0.005287212, 0.00576672, 0.03407034, -0.010208483, -0.0029921958, -0.048632253, 0.010202173, -0.03598837, 0.014990948, 0.0014361594, -0.015041423, 0.016416855, 0.059004776, -0.0040537394, 0.00488657, -0.036770727, 0.02344544, 0.005123169, -0.029325727, 0.026171068, -0.0032713835, -0.00479193, 0.0018454767, 0.0017729194, 0.003713036, 0.019104626, -0.035710763, 0.011356779, 0.02166621, -0.016126627, -0.016416855, 0.010725847, 0.011003457, 0.006902398, 0.005135788, -0.042625777, 0.022435948, -0.03770451, -0.046108525, -0.01925605, -0.015975203, 0.026650576, 0.008416635, 0.010031821, 0.049666982, -0.002506378, -0.017539915, -0.008713174, -0.0057067815, -0.018271796, -0.013464092, -0.003914934, 0.016732322, -0.0076784445, -0.03608932, 0.012107588, 0.0004657068, 0.0019590445, 0.010549186, 0.031546608, 0.016517805, 0.028265761, -0.01162808, 0.0065806224, 0.029426677, 0.021199321, -0.0075270208, -0.010807868, 0.04578044, 0.04903605, -0.0015221239, 0.019748177, 0.007394525, -0.008366161, -0.007823559, 0.018208703, -0.027104847, -0.007375597, -0.012568169, -0.02557799, -0.011994021, 0.018991059, 0.04315576, 0.0016830115, -0.0034827457, -0.010195864, 0.0150540415, -0.004993828, 0.004801394, -0.03033522, -0.016202338, 0.006763593, 0.02018983, 0.0060727224, -0.0019211884, 0.000928259, -0.06965491, -0.02801339, 0.010252648, -0.026019644, 0.019571517, -0.01571021, -0.017918473, -0.000544179, -0.03990015, -0.014524058, 0.011735339, -0.012549241, 0.0136028975, 0.03142042, -0.003432271, 0.01162177, -0.007135843, -0.0051137055, -0.038991608, 0.03248039, 0.0017871154, 0.045174744, -0.013072914, 0.005154716, 0.00016946445, 0.0006273832, -0.009785758, 0.00041444358, 0.0280891, -0.015533551], "content_hash": "f98ab795e8dbf01a5cffac7402bc2125"}, {"content": "(BoW) Heads **What they do**: These heads spread their attention **broadly and evenly** across many tokens, without focusing strongly on any particular one. **Why they matter**: BoW heads capture a general summary of the entire input. They help the model maintain an overall sense of what the text is about.\n\n**Detection**: Classified when the attention distribution has high entropy (\u2265 0.65 normalized) and no single token receives more than 35% attention.\n\n### Syntactic Heads\n\n**What they do**: These heads attend to tokens at **consistent distances**, suggesting they track grammatical or structural relationships (like subject-verb pairs).\n\n**Why they matter**: Syntactic heads help the model understand grammar and sentence structure. They might connect a verb to its subject or a pronoun to what it refers to.\n\n**Detection**: Classified when tokens consistently attend to other tokens at similar distances, with low variance in attention distances.\n\n### Other\n\n**What they do**: Heads that don't clearly fit any of the above patterns. They may have mixed or context-dependent behavior.\n\n**Why they matter**: \"Other\" doesn't mean unimportant. These heads may serve specialized roles that only activate for certain inputs. They're worth investigating through ablation experiments.\n\n## Using Categories for Experiments\n\nHead categories are especially useful for guiding ablation experiments:\n- Ablate a **Previous-Token** head to see if local context patterns break\n- Ablate a **BoW** head to see if the model loses global context\n- Compare the effect of ablating heads from different categories on the same prompt", "source_file": "head_categories_explained.md", "chunk_index": 1, "embedding": [0.019570751, 0.042948492, 0.03524991, 0.038351905, -0.023307241, -0.044104688, -0.021657545, -0.026804032, 0.0064542587, 0.05623066, 0.017286556, 0.0014637526, -0.040241297, 0.014762663, 0.0010945097, 0.00597486, -0.008318979, 0.020656448, 0.021911345, 0.05555386, 0.007180407, 0.0049737627, 0.014205715, 0.05205707, 0.06553663, 0.012217619, -0.020021949, 0.040495098, 0.018329954, 0.010603174, 0.013789766, -0.022559943, -0.050534274, -0.028651128, -0.0394235, 0.018259455, -0.02402634, -0.003613116, 0.02777693, 0.021967744, 0.015326661, -0.022827843, -0.07190982, 0.030089324, -0.03539091, -0.0046882383, 0.033755314, 0.004684713, -0.012083669, 0.053467065, -0.041482095, 0.061983444, 0.0030967046, 0.0050724624, 0.0154112615, 0.019598952, -0.0393107, 0.017357057, 0.02786153, -0.02411094, 0.0046036383, -0.0017008082, 0.005601211, 0.02773463, 0.0020568324, -0.057386857, -0.05938905, -0.047263082, 0.009827675, -0.014114064, 0.081384994, 0.031499323, 0.02422374, 0.013832065, -0.005178212, -0.02027575, 0.0046882383, -0.015101062, -0.033642516, -0.020360349, 0.00301563, 0.02753723, 0.012443218, -0.0395645, -0.010962723, -0.04486609, -0.03135832, -0.020430848, -0.038351905, -0.04348429, -0.026268234, 0.024985136, 0.0060770847, -0.012140069, -0.0397901, -0.032627318, 0.01204842, -0.033134915, 0.017554456, -0.0024163814, -0.0010821723, -0.01930285, -0.03242992, -0.012471419, 0.056653656, -0.046811882, -0.001731652, -0.041961495, -0.0399029, -0.02030395, -0.07478621, -0.007109907, -0.005604736, 0.01594706, 0.04723488, -0.030512324, -0.02425194, 0.022729144, 0.010025075, -0.015481761, 0.029750925, -0.013627616, 0.010278874, -0.009101527, 0.00092707266, -0.0025626686, 0.019119551, -0.03265552, -0.00050759874, 0.006461309, 0.036857307, 0.0054707862, 0.01966945, -0.041482095, -0.029440725, -0.0046600383, -0.018738853, 0.04850388, -0.07128942, 0.0092566265, 0.020346249, -0.00040801772, 0.027128331, 0.011096672, -0.022249743, 0.0038457653, 0.0035390912, 0.038323704, -0.016539259, 0.04337149, -0.011160122, -0.02402634, -0.048955075, -0.012471419, -0.010723023, -0.034911513, 0.015524061, 0.027593631, -0.006221609, -0.053805463, 0.028707528, 0.013000167, -0.002515081, 0.013606465, -0.029074127, -0.0037999405, -0.012605368, -0.009531576, -0.040861696, -0.009016927, -0.018104354, 0.0037047656, -0.019359251, -0.012675868, -0.0010997972, 0.014678063, -0.025041537, -0.017159658, 0.0033698916, -0.022503544, 0.038944103, 0.035983108, -0.013000167, -0.02360334, -0.02807303, -0.014198665, -0.0023441191, 0.019472051, -0.05036507, 0.0036800907, -0.01955665, 0.040720697, 0.08787098, 0.038492903, 0.018033855, -0.042328093, 0.014396064, 0.040043898, 0.019274652, -0.022263844, 0.022574043, 0.00094117265, 0.03603951, -0.00790303, 0.0015095775, 0.026395133, 0.019570751, 0.03601131, -0.042017896, -0.04418929, 0.00817093, -0.08527659, 0.033050317, 0.046417084, -0.07416581, -0.0399593, -0.073827416, 0.02350464, 0.0047481633, 0.010229524, 0.04754508, 0.01973995, 0.066664636, 0.05995305, -0.04292029, 0.021770345, 0.04506349, -0.018922152, -0.0153548615, -0.021869045, 0.049265277, -0.0012275781, 0.009623226, -0.0026789932, -0.06367544, -0.09119857, 0.022404844, -0.013409066, 0.05233907, -0.045937683, -0.011181272, -1.7748882e-05, 0.019133652, 0.00817093, -0.0017765955, 0.018301753, -0.010610224, -0.05893785, 0.01931695, 0.026338734, 0.08708138, -0.00026195092, -0.07760621, 0.018851653, -0.047037482, 0.004875063, 0.009080377, 0.03595491, -0.012718168, 0.03527811, -0.045486487, -0.0010539724, 0.003528516, 0.017159658, -0.037816104, -0.045176286, -0.0043956637, -0.012358619, -0.01591886, -0.019288752, -0.052395467, 0.008882978, 0.045824885, -0.01985275, -0.038239103, 0.005199362, -0.02412504, -0.01988095, -0.04438669, -0.017216057, 0.021417847, -0.0047164382, 0.057133056, 0.0075082313, -0.044330288, 0.0070746574, -0.026014434, -0.012753418, 0.012936718, 0.016680257, 0.009869975, -0.017328857, 0.04506349, -0.033304118, -0.0011870408, -0.010899273, 0.034544915, 0.030173924, 0.046868283, -0.0023088693, 0.046417084, -0.037364908, 0.033388715, 0.01954255, 0.03499611, -0.00784663, -0.00013042467, -0.005957235, -0.015058762, -0.010363474, -0.029045926, -0.013782715, -0.0112306215, -0.014142265, -0.06311144, -0.05848665, 0.020134749, 0.030117525, -0.000339721, -0.0016884707, -0.06711583, -0.034403913, 0.021826746, -0.01621496, 0.039818298, 0.01199907, -0.017498055, 0.030878922, -0.04889868, -0.024886437, 0.038210902, 0.026056735, 0.013465466, 0.016905857, -0.033980913, -0.04438669, 0.012887368, 0.03152752, -0.026860433, 0.037393104, -0.05236727, 0.012140069, -0.01587656, 0.024576237, -0.021826746, 0.06649543, -0.005608261, 0.008361279, -0.0068843076, 0.0039092153, -0.017032757, -0.009989825, 0.013902565, -0.013691066, -0.022954741, 0.06581864, -0.016623858, -0.03674451, 0.02368794, 0.0066340333, -0.009728976, -0.014960063, 0.014269164, 0.025633736, 0.018033855, -0.02385714, -0.01992325, -0.014198665, 0.005551861, 0.023067541, -0.015256162, 0.015848361, -0.026352834, -0.033811715, -0.05702026, -0.0040149647, -0.018936252, -0.003165442, -0.029497126, 0.049067877, -0.028989527, -0.004416814, -0.05575126, -0.03572931, 0.026677134, 0.014304414, -0.0050759874, 0.013366766, -0.03609591, 0.0069794822, -0.020670548, 0.021713946, 0.008812478, 0.026634833, -0.025901634, 0.03496791, -0.004332214, 0.05484886, 0.031048123, 0.039169703, -0.0062180846, 0.012147119, -0.05513086, -0.011301122, 0.025943935, -0.0004095599, 0.019218251, 0.009172027, 0.017286556, 0.0061687343, 0.06457784, 0.05617426, 0.020571848, 0.016680257, -0.0131482165, 0.015002362, -0.012203519, 0.017610855, 0.036434308, 0.028340928, 0.022559943, -0.06361904, -0.0396773, -0.0053474116, -0.042328093, -0.004353364, 0.01976815, -0.06683383, 0.02804483, -0.06367544, -0.03288112, 0.056710057, 0.02389944, -0.022630444, 0.014720363, -0.058655854, 0.032147918, 0.054482263, 0.01962715, -0.005090087, -0.010969772, -0.020430848, -0.0056117857, -0.0066093584, -0.025069738, 0.00791008, -0.010504474, -0.045824885, 0.009468126, -0.061983444, 0.02017705, 0.010751223, 0.007275582, 0.026789933, -0.003965615, 0.003532041, 0.04461229, -0.017836455, 0.04399189, -0.02817173, 0.023250842, -0.010292974, -0.013430216, -0.0025802935, -0.010708923, 0.02821403, -0.022418944, -0.022207445, 0.051126473, -0.018795252, 0.045514684, 0.02819993, 0.024181439, -0.04715028, 0.011040272, -0.038239103, 0.016496958, -0.0047481633, 0.01175232, -0.0393107, 0.032740116, -0.02745263, 0.011287021, 0.010828773, 0.01920415, 0.038436502, 0.0049032625, -0.009827675, 0.012485519, -0.01576376, -0.02721293, -0.012689969, 0.015368962, -0.016919957, 0.032119717, -0.0055589112, -0.012957867, 0.008192079, 0.037534107, -0.014290314, 0.050252274, -0.01586246, 0.008579829, 0.04308949, 0.023335442, 0.013472516, 0.030258523, 0.015481761, -0.03203512, 0.02787563, -0.021784445, -0.014861363, -0.0013738653, 0.048785876, -0.01180872, 0.010694823, 0.018936252, 0.02841143, 0.009016927, 0.029327927, -0.015171562, 0.034883313, -0.003435104, -0.012725218, 0.010687773, -0.029666325, -0.015566361, -0.037139308, -0.028933126, 0.0021185197, 0.003221842, -0.048588477, -0.004494364, 0.0009658476, -0.03547551, -0.014036515, -0.007874831, -0.025041537, -0.021192247, 0.024773637, 0.025238937, -0.007585781, 0.018062055, -0.020797448, 0.020867947, -0.006753883, 0.031189121, -0.01222467, 0.031471122, 0.036180507, -0.019472051, -0.034375712, 0.037364908, -0.02315214, -0.015298462, 0.010976822, -0.041651294, -0.039169703, 0.0037364906, -0.025661936, 0.024167338, -0.055215463, 0.0053333114, -0.018428653, -0.0042299894, 0.026691234, -0.014212764, 0.0032852918, 0.017201956, -0.015439461, -0.0394799, -0.027043732, 0.021812646, -0.0410309, 0.017836455, -0.004279339, 0.013592366, 0.011181272, -0.0004996675, 0.003542616, -0.01941565, -0.025041537, -0.020882048, 0.01190037, -0.020656448, -0.0049490877, 0.027396232, 0.01930285, -0.019006751, 0.009996875, 0.010483324, -0.024280138, 0.021995945, 0.024886437, -0.015368962, -0.0047164382, 0.020839747, 0.015467661, -0.024914637, 0.0128591675, 0.014121114, 0.013197566, 0.009390577, 0.032937516, -0.021699846, -0.0073460815, -0.032373518, -0.027368031, 0.016722558, 0.025958035, -0.0047199633, 0.026056735, -0.033332318, -0.017921055, -0.029440725, 0.031301923, -0.009164977, -0.030822523, 0.009108577, -0.031471122, -0.0114068715, 0.0009896413, 0.01207662, 0.004445014, -0.0014769713, 0.0408053, 0.029807325, -0.037393104, 0.029130526, 0.002788268, -0.054454062, -0.0002546806, 0.024660839, -0.0050759874, -0.017667256, 0.004924413, -0.011216522, -0.029412527, 0.02380074, 0.024308339, -0.025718335, 0.0020039575, -0.022376643, -0.0023300191, -0.01180872, -0.018090254, -0.0056611355, 0.0041489145, -0.010405774, -0.019218251, 0.020698749, -0.029835526, 0.05995305, 0.034685913, -0.0033945665, -0.012400919, -0.016370058, 0.017385256, -0.016313659, -0.012492568, -0.025394036, 0.007116957, -0.014022415, -0.046219684, 0.0005816235, 0.0068631577, -0.007465931, 0.00805108, 0.0007658043, -0.017639056, -0.058091853, 0.03220432, 0.07450421, 0.021192247, 0.03674451, 0.029186927, 0.0010795286, -0.0044732136, -0.02003605, 0.037336707, -0.021742145, 0.049039677, -0.03603951, -0.048842277, 0.010046225, -0.005488411, 0.02349054, -0.012464369, 0.0075082313, 0.0057986104, 0.021643445, -0.02337774, 0.015552261, -0.005971335, 0.008615078, -0.023307241, -0.016426459, 0.021305047, -0.021502446, -0.041059095, 0.0072685317, 0.0031460547, -0.008424729, -0.013733366, -0.0038034655, -0.01996555, 0.011653621, 0.054284863, 0.0064295838, -0.023279041, -0.042440895, 0.022700943, 0.00021425336, 0.04810908, -0.008396529, 0.020656448, -0.013225767, -0.026747633, 0.005562436, -0.0101872245, -0.013662865, 0.00079885114, -0.015002362, 0.018386355, 0.030399524, 0.0040184897, 0.0014514151, -0.026973233, -0.029074127, -0.023011142, 0.00401144, 0.021248646, -0.00596076, -0.03587031, -0.0012548969, 0.038464703, -0.041707695, 0.016313659, 0.021333246, -0.020233449, 0.053946465, -0.021008948, 0.0041348147, -0.012697018, 0.018160755, 0.00392684, 0.011153072, 0.025323536, -0.030004725, -0.0032447544, 0.003651891, 0.013338567, -0.010744173, -0.01206957, 0.026240034, -0.02019115, -0.016595658, -0.008798378, 0.01598936, 0.03226072, -0.0116959205, 0.021671645, -0.020783348, -0.009249576, -0.015086962, -0.03505251, -0.016285459, -0.021262746, -0.010342324, 0.019373352, -0.015495861, 0.01982455, 0.038915902, -0.017526256, 0.0395081, -0.01995145, 0.0397901, -0.03178132, -0.030935323, 0.00014342308, 0.022277944, -0.008932328, -0.017244257, 0.0024128563, -0.0047481633, 0.01558046, -0.031104522, -0.022545842, 0.009235477, -0.05871225, 0.00040427243, 0.015016463, 0.009214327, -0.025676036, -0.018273555, -0.00811453, -0.02409684, -0.09509016, -0.009284827, 0.030737923, -0.030399524, -0.0390851, 0.008347179, 0.034206513, 0.037900705, 0.011216522, 0.00801583, 0.014522963, 5.7666653e-05, -0.013521866, -0.030343125, 0.042158894, -0.053269666, -0.018654253, 0.0033452166, 0.0013016029, -0.020529548, -0.017145557, 0.015143362, -0.010807623, -0.010208374, -0.050308675, 0.015129262, -0.017173756, -0.0031777795, 0.0060418346, 0.01613036, -0.018710652, 0.015383061, -0.024407038, 0.023631541, -0.01579196, 0.02394174, 0.017300656, 0.0061264345, 0.034234714, -0.027170632, 0.02030395, -0.025535036, 0.006278009, -0.005157062, -0.0060770847, 0.005058362, 0.00779023, -0.020261649, 0.0006851702, 0.022517644, -0.0017193144, -0.01999375, 0.012386818, 0.030596923, 0.012302219, -0.034657713, -0.033501517, -0.013493666, 0.024999237, 0.01191447, 0.010109674, -0.0072121317, 0.024604438, -0.0062603843, -0.020459048, -0.012203519, -0.0025503312, 0.033473317, -0.040043898, -0.0068913577, -0.016708458, 0.007536431, 0.0116395205, 0.015101062, 0.03206332, -0.0062286593, 0.004734063, 0.01965535, -0.014057665, -0.0012663531, 0.06542384, -0.034460314, -0.014156365, 0.048842277, 0.016637959, 0.021671645, 0.0048609627, -0.0015703836, 0.009284827, 0.02752313, -0.00788188, -0.04712208, 0.0044520637, 0.03144292, -0.067454234, -0.017469857, 0.017723655, 0.015368962, 0.05180327, 0.00043974264, 0.008530479, 0.004437964, 0.018358154, -0.03479871, -0.041679494, -0.015467661, -0.0052945367, -0.0033610791, -0.02050135, -0.0073249317, 0.027339831, 0.023645641, 0.009510426, 0.023039343, -0.004381564, -0.0077056307, 0.024392938, 0.013359716, 0.005513086, -0.0021978319, -0.043879088, 0.025295336, -0.008608028, 0.001326278, 0.013077717, 0.007169832, -0.017159658, -0.013726315, 0.010793523, -0.01608806, -0.0064295838, -0.017526256, -0.029750925, -0.017018657, -0.001959895, -0.005597686, 0.000889179, 0.006362609, -0.0032764792, 0.026804032, -0.009693726, -0.036857307, 0.056766458, 0.014085865, 0.013980115, -0.0024146189, -0.03178132, -0.0396491, -0.0061017596, 0.015552261, -0.005329787, 0.01174527, 0.0013747466, -0.005971335, -0.0019299326, -0.00063361716, 0.030766122, -0.0038915903, -0.010983872, 0.06378824, -0.010589073, 0.010264774, 0.05118287, -0.018033855, 0.024646739, 0.026592534, -0.054482263, 0.0004952613, 0.0141352145, -0.0036836157, -0.038210902, -0.01645466, 0.023053441, 0.048616678, 0.017935155, 0.026987333, -0.025972135, -0.022179244, -0.0069830073, 0.011364572, 0.02382894, -0.0012073094, -0.028876727, -0.019246452, 0.014776763, 0.0392543, 0.0006552077, 0.0037999405, 0.0046670884, -0.013430216, 0.018654253, 0.04630428, -0.023391841, -0.020430848, -0.038464703, -0.026254134, 0.018640153, 0.01962715, -0.007874831, -0.017441656, -0.022827843, 0.034009114, 0.0030808423, -0.033501517, -0.0049984376, 0.03584211, -0.012936718, -0.0042828643, 0.009919325, 0.006746833, -0.016623858, 0.037364908, -0.029694526, 0.004822188, -0.037618704, -0.018231254, 0.025267137, 0.007064082, 0.038126305, -0.0030262049, -0.017540356, -0.01206957, -0.02798843, 0.038069904, 0.013063617, 0.0031636795, 0.045035288, 0.012217619, 0.015636861, -0.016821258, -0.038098104, -0.010617273, -0.0037223906, 0.024618538, -0.007081707, -0.009045127, -0.010511524, 0.010715973, -0.018217154, -0.01603166, 0.025154337, 0.018301753, -0.028651128, -0.01179462, -0.0030473548, -0.030117525, 0.026211834, -0.00034919442, 0.009644376, -0.013832065, 0.007571681, 0.0005847079, 0.03634971, -0.020557748, -0.0018435704, -0.026042635, 0.01999375, 0.025436336, 0.052480068, 0.022898342, 0.03206332, 0.022588143, -0.0012681156, -0.033811715, -0.010758273, 0.03524991, -0.01603166, 0.001575671, -0.002148482, -0.015622761, -0.021643445, 0.03457311, -0.010095575, 0.0057527856, -0.026338734, 0.003200692, -0.008685579, 0.0021995944, -0.0039127404, -0.013705165, 0.026536133, 0.012295169, 0.002798843, 0.012358619, -0.01988095, -0.01194267, -0.028326828, -0.021643445, 0.018301753, -0.013296267, -0.006708058, -0.010321174, 0.00041947394, 0.0043110643, -0.04396369, -0.007557581, 0.014283264, 0.016468758, 0.021826746, -0.026409233, 0.0015721461, 0.010116724, 0.02829863, -0.031019922, 0.0069653825, -0.013733366, 0.016172659, -0.061927043, -0.030371323, 0.0017184332, 0.0064859837, 0.017272457, -0.0005349174, 0.037477706, -0.029497126, -0.009221377, 0.007331982, -0.053692665, -0.005604736, -0.048165478, 0.0071275323, 0.0402977, -0.017709555, 0.013796816, -0.002529181, -0.024717238, 0.007190982, -0.003410429, -0.018485054, 0.008953477, -0.013648766, -0.0027071931, 0.01211892, 0.0077690803, -0.0399593, 0.042074293, -0.02808713, -0.0042088395, 0.022968842, 0.004896213, -0.018950352, 0.03166852, 0.007522331, 0.014064714, 0.015650962, 0.026973233, -0.00795943, -0.010335274, -0.00076756685, 0.00812158, 0.081497796, -0.02350464, -0.037308507, 0.0034527287, -0.0016259021, 0.014988262, -0.010906323, -0.028552428, -0.03499611, -0.013317416, -0.036800906, 0.017864656, 0.0013738653, -0.0006922201, -0.011054372, 0.015185662, -0.012083669, -0.0041982643, 0.031696722, -3.4313587e-05, 0.013183467, 0.0400157, -0.014551164, -0.00202687, 0.008361279, -0.00611586, 0.032345317, 0.0069160326, -0.0019810451, -0.004649463, -0.016313659, -0.037928905, -0.030850723, -0.014304414, -0.01610216, 0.017695455, -0.004346314, -0.0022683318, -0.0034950287, -0.006838483, -0.018738853, 0.033614315, 0.0058268104, 0.003005055, 0.01610216, 0.006221609, 0.019697651, 0.030343125, -0.005435536, -0.015185662, -0.03550371, 0.0056364606, 0.0028358554, -0.0007437731, 0.01971175, -0.045627486, 0.010567923, 0.029327927, 0.014917763, 0.022292044, 0.017032757, 0.054059263, -0.021784445, 0.007635131, 0.000490855, -0.012675868, 0.00074068876, -0.008375379, 0.008939378, 0.0033769414, -0.017455757, -0.042328093, 0.02382894, 0.0010231286, -0.006729208, -0.013839115, -0.024872338, -0.02422374, 0.017103257, -0.004360414, 0.009390577, -0.028904928, -0.054707862, 0.01601756, -0.00807928, -0.017328857, 0.0077690803, -0.008939378, -0.03646251, 0.022489443, 0.005706961, 0.032852918, 0.028735727, 0.04337149, 0.017328857, -0.010433974, -0.013303316, -0.01589066, 0.026789933, -0.025802935, 0.01644056, -0.016652059, -0.034206513, 0.014861363, -0.027480831, -0.01168182, -0.035080712, 0.019232351, -0.01192152, 0.007592831, 0.009418776, -0.020120649, -0.00812863, 0.011554921, -0.00017173316, 0.0131482165, -0.013098867, 0.045091685, -0.014790863, -0.030455923, 0.024759538, -0.019091353, -0.009031028, -0.0054531614, -0.0139589645, 0.0043392642, 0.03133012, 0.007825481, -0.025323536, 0.011336371, -0.010095575, 0.021502446, 0.0086221285, -0.011569021, -0.014353764, -0.014819062, 0.03116092, 0.007155732, -0.045373686, 0.0023705566, 0.010257725, 0.009728976, -0.009468126, -0.018767053, 0.008340129, -0.01173822, 0.024209639, 0.025492735, -0.022418944, 0.002404044, -0.014212764, 0.009023977, 0.020726947, 0.009291877, -0.008347179, -0.03265552, 0.011082572, 0.0016417647, -0.01202022, -0.0107653225, 0.028580628, 0.0064472086, -0.0014813775, 0.0040960396, 0.01975405, 0.046219684, -0.0050513125, -0.0032412293, 0.032345317, 0.0031143297, 0.030371323, -0.0029098801, -0.018611953, 0.0098417755, -0.01174527, -0.0036977157, 0.0017404644, 0.00816388, 0.00090592273, 0.0026032059, -0.050844472, 0.018456854, -0.012217619, -0.030343125, 0.01608806, 0.0062850593, -0.018245354, -0.006849058, 0.022277944, 0.017836455, 0.009454026, -0.032965716, -0.021446045, -0.00600306, 0.021685746, 0.053410664, -0.00395504, -0.009531576, 0.017878756, -0.021502446, 0.013141167, -0.021023048, -0.0089675775, -0.0019810451, -0.006299159, -0.00063185464, 0.011533771, 0.007684481, 0.013268067, 0.008974628, -0.0033170166, 0.0052099368, -0.0037329656, -0.0009041602, 0.014974162, 0.03550371, -0.015256162, 0.0111178225, -0.021135846, -0.030907122, -0.019373352, 0.013514816, 0.050252274, -0.0032200795, -0.035560112, 0.038774904, -0.027353931, -0.02741033, -0.03584211, -0.010116724, 0.00035756628, -0.03161212, -0.013620566, 0.025901634, 0.020459048, 0.004388614, -0.017371157, 0.023222642, 0.0012892655, 0.010553824, 0.034150112, 0.041397497, -0.037731506, 0.008565729, -0.014304414, 0.027579531, 0.0061722593, -0.025605535, -0.012527819, -0.00028706648, 0.0005895548, 0.020712849, -0.03589851, -0.019260552, -0.014988262, -0.026663033, 0.017808255, -0.011209472, 0.0069759577, 0.0061616846, -0.086009786, 0.0044485386, 0.0021467195, 0.011618371, -0.0040466897, 0.040889896, -0.011005023, -0.0021361446, -0.030794322, 0.010997972, 0.0045084637, 0.0057633603, -0.0092002265, -0.017765956, 0.004272289, 0.048842277, 0.022503544, 0.021460146, -0.013472516, 0.006789133, 0.014184564, -0.028665228, 0.010483324, 0.022066444, -0.012520769, 0.01655336, 0.01614446, -0.005090087, 0.01194972, -0.0038633903, 0.000601011, 0.008283729, -0.013810915, -0.010899273, 0.0020920823, 0.017963355, 0.006796183, 0.0046952884, -0.049011476, -0.00028111803, -0.027142432, -0.0404105, -0.020952547, -0.029017728, 0.0051006624, 0.02794613, -0.008495228, 0.06085545, 0.00015267618, -0.003979715, -0.0065459087, -0.0046952884, -0.033050317, -0.014720363, -0.007628081, -0.0005886735, 0.004850388, -0.009066277, -0.015171562, -0.006355559, -0.011646571, -0.0019810451, 0.02839733, 0.026789933, 0.0129155675, -0.018908052, -0.01200612, 0.0066692834, 0.0028711052, 0.010539724, -0.008199129, 0.03186592, 0.029468926, -0.025718335, 0.021037146, -0.031104522, -0.009038078, -0.0014214527, 0.016708458, -0.029581726, -0.009764225, -0.020853847, -0.026268234, -0.0038175655, 0.008304879, 0.056710057, 0.0076421807, 0.009714875, -0.016496958, 0.0076915305, 0.0019369826, -0.0052874866, -0.015213862, -0.017751856, -0.011773471, 0.028524227, 0.014142265, -0.023138043, 0.0107653225, -0.068751425, -0.04486609, 0.016948158, -0.024392938, 0.021544745, 0.0030068175, -0.019345151, -0.012259919, -0.034291115, -0.0060488847, -0.00408899, 0.0092566265, -0.013197566, 0.029525327, -0.015002362, -0.01655336, -0.0018999702, -0.0033311166, -0.021065347, 0.03601131, -0.017963355, 0.041623097, -0.018654253, 0.031189121, 0.0011606034, -0.00016776755, 0.0086785285, -0.0006785608, 0.02367384, -0.0131482165], "content_hash": "7515536ff8f44ba0e57b7bb55a968995"}, {"content": "# Interpreting Ablation Results\n\n## Quick Reference\n\nWhen you ablate an attention head and see the results, here's how to interpret what happened.\n\n## Probability Changes\n\nThe dashboard shows the immediate next-token probability before and after ablation (e.g., \"72.3% \u2192 45.1% (-27.2%)\").\n\n### Large Probability Drop (>10%)\n\nThe ablated head was **important** for this prediction. It was actively contributing to the model's confidence in the top token. This head likely plays a significant role in processing this specific input.\n\n**Example**: Ablating a Previous-Token head when the model is predicting a word that commonly follows the previous word (like predicting \"the\" after \"on\").\n\n### Small Probability Drop (1-10%)\n\nThe head has **some contribution** but isn't critical. Other heads or MLP layers may provide overlapping information. The model has some redundancy that compensates for the missing head.\n\n### Negligible Change (<1%)\n\nThe head was likely **redundant for this input**. It may serve a function that isn't relevant to this particular prompt, or other heads provide the same information.\n\n**Important**: This doesn't mean the head is useless -- it might be critical for other prompts. Try the same head with different inputs.\n\n### Probability Increase\n\nOccasionally, ablating a head can **increase** the probability of the top prediction. This means the head was actually pulling the model away from this prediction -- it was a \"competing signal.\" This is an interesting finding that suggests the head was promoting a different output.\n\n## Generation Changes\n\nThe full generation comparison shows whether the ablated model produces different text.\n\n### Generation Changed\n\nThe head was important enough that removing it altered the model's entire output sequence. This is a strong signal of importance. Look at where the texts diverge -- the point of divergence tells you where the head's contribution was most critical.\n\n### Generation Stayed the Same", "source_file": "interpreting_ablation_results.md", "chunk_index": 0, "embedding": [0.02914487, 0.013198973, 0.04447271, 0.03246865, -0.028032366, 0.009813388, -0.003222486, 0.01476472, 0.017182013, 0.046093397, 0.009167861, -0.032303832, -0.06202556, 0.035270512, 0.025367849, 0.0036259405, 0.033704765, 0.008639078, -0.007835602, 0.04329153, -0.027180819, -0.029254748, 0.021563359, 0.053537562, 0.024255345, -0.02955691, 0.022085274, -0.0021065478, 0.02567001, 0.014668577, -0.00450839, -0.01731936, -0.019667981, -0.02171444, -0.05425176, 0.00027361946, 0.0054423446, 0.009284605, -0.01928341, -0.0131509015, 0.012793802, -0.029282216, -0.031891797, 0.04444524, -0.025079422, 0.049417175, 0.045516543, -0.059278633, -0.041341215, 0.054993432, -0.016797444, 0.048620567, -0.034418967, 0.008934372, -0.020890363, -0.0010532739, 0.0068364087, -0.0058921534, 0.010616864, -0.052273978, -0.020176161, -0.0050337394, 0.060981728, 0.020533262, -0.0029168907, -0.032715872, -0.027139615, 0.016701302, 0.010582527, -0.026686372, 0.04345635, -0.017209483, 0.0033666994, 0.009339544, -0.02620566, -0.023362594, -0.0036980473, -0.011523349, -0.05801505, -0.02109638, 0.033677295, 0.023802102, 0.032770813, -0.03571002, 0.0071214023, -0.048730444, -0.01714081, -0.014229069, -0.010733608, -0.05977308, -0.016975995, -0.0012721695, 0.014902066, 0.00056097354, -0.034748595, 0.01622059, 0.004896393, -0.015588798, -0.021206258, 0.029584378, 0.0063144933, -0.0055831242, -0.004927296, -0.022552252, 0.027661532, -0.026480354, -0.04623074, -0.00047641975, -0.0026250298, -0.005703302, -0.019613042, -0.04452765, -0.038456947, 0.032221425, -0.02388451, -0.038072377, -0.044939686, 0.018528007, 0.026068315, -0.031699512, -0.024873402, -0.012725129, 0.0011022035, 0.0014816225, -0.0069703213, -0.0006326511, 0.04051714, -0.016206855, -0.008007285, -0.021590827, 0.031946734, 0.017374298, -0.0106512, -0.045022096, -0.067958914, -0.008151499, 0.024076795, 0.032880686, -0.057685416, 0.02477726, 0.011022035, -0.013954377, -0.024639914, -0.0068467096, -0.075925, 0.021316135, 0.026356742, 0.030353518, -0.023925714, 0.016083244, 0.007464768, 0.011866715, -0.007972949, -0.025793623, -0.006376299, -0.03389705, -0.0061428105, 0.014490028, -0.0023245849, -0.06488236, 0.027276961, 0.047137227, -0.027702736, -0.01708587, -0.025738684, 0.0358199, -0.01589096, 0.014462559, -0.02881524, -0.064717546, -0.04373104, -0.020698078, -0.018253315, -0.034089334, -0.033375133, 0.00030409315, -0.061311357, -0.010870954, 0.009882062, -0.03941837, -0.0031933, 0.013198973, -0.03535292, 0.030490864, -0.0072381464, -0.017511645, -0.03573749, 0.033375133, -0.039885346, -0.017072137, 0.023046698, 0.0049616327, 0.054498985, 0.006613221, -0.023417532, -0.028059835, 0.07587006, 0.008096561, -0.007320554, 0.0047659143, 0.036314342, 0.023939447, 0.018967515, -0.0019297146, 0.023953183, 0.010850352, -0.0070595965, 0.004237131, -0.03098531, -0.0358199, 0.017841276, -0.06900275, 0.01565747, 0.0049925353, -0.030023886, -0.013679685, -0.052686013, 0.058894064, 0.042522393, 0.025848562, 0.024584975, 0.013604145, 0.018225845, 0.048263468, -0.016193122, 0.028732833, 0.064607665, 0.0022919653, 0.015396513, -0.031150125, 0.010994566, 0.029282216, -0.0028173146, -0.019338349, -0.0020979636, -0.08152872, 0.04370357, -0.01639914, 0.032221425, -0.015066882, 0.0049719336, 0.0030765557, 0.048181057, 0.045791235, 0.033320196, 0.023994386, -0.022387436, -0.04263227, 0.024997015, 0.061366297, 0.028513078, 0.009202197, -0.044994626, 0.02207154, -0.02620566, -0.020286039, -0.02914487, 0.002492834, -0.021700704, 0.025093157, -0.02572495, 0.0044465843, -0.045434132, -0.0016807745, -0.0262606, 0.0002882125, 0.00053049985, 0.013253911, -0.07361758, -0.03395199, -0.029776663, -0.02370596, 0.005160785, 0.011736236, -0.033842113, -0.028513078, -0.024763526, 0.009827123, -0.03579243, 0.008384988, -0.013981846, -0.02334886, 0.03104025, -0.011626358, -0.024708588, 0.0043435744, -0.01696226, -0.024873402, 0.0031177595, -0.008934372, -0.0006609787, -0.01580855, 0.056366894, -0.022593455, -0.020533262, 0.006716231, 0.02299176, 0.020450855, -0.03428162, 0.018431865, 0.031672042, -0.026700107, 0.0077119907, 0.0037941898, 0.04090171, -0.0013768959, 0.028513078, 0.020849159, -0.024736056, 0.031562164, -0.018321987, 0.011798041, -0.024845934, -0.036259405, -0.06740953, -0.0180885, 0.048235998, -0.033704765, -0.0055144513, -0.0075265737, -0.005528186, -0.062190372, 0.002204407, 0.004607966, -0.01747044, 0.021810582, -0.0061084735, -0.0065102116, 0.0031538128, -0.02712588, 0.07658426, 0.016261794, 0.013714021, 0.037468053, -0.052054223, 0.015369044, -0.0422477, 0.04837334, 0.0052363253, 0.02720829, -0.043511286, -0.008330049, -0.018967515, 0.004072316, -0.015437717, 0.0017700496, -0.008082826, 0.013933775, 0.027661532, 0.02186552, -0.032496117, 0.014119193, 0.015272901, -0.027043473, 0.01246417, 0.07416697, -0.0016138182, -0.009442554, 0.013459931, -0.003351248, -0.02735937, 0.0023692225, 0.0023537711, 0.009250268, 0.015918428, -0.023664756, -0.050186314, 0.05068076, 0.013006688, 0.0041856263, -0.018692823, -0.009408217, -0.012203213, -0.019146064, -0.10581154, -0.023609817, -0.032578528, -0.013659083, -0.035078228, 0.037989967, 0.018582946, -0.016110713, -0.024008121, -0.05318046, 0.028458139, 0.011042637, -0.02881524, -0.011592022, -0.07938612, 0.038868982, 0.030380987, -0.013164637, -0.007073331, 0.026713843, -0.020698078, 0.06312433, -0.045434132, 0.020835424, 0.037440583, 0.05801505, 0.00773946, -0.0019417324, -0.04743939, -0.02955691, 0.01663263, 0.017635256, 0.0016447211, -0.00081978535, 0.02439269, 0.026425416, 0.067189775, -0.00592649, 0.033072975, -0.0393909, 0.0390338, -0.003649976, 0.060322464, 0.052740954, 0.009181595, 0.011832378, -0.020890363, -0.080759585, -0.015877225, -0.03213902, -0.04444524, -0.026864924, 0.006135943, 0.0077669295, 0.05834468, -0.068068795, -0.025381584, 0.032715872, 0.051916875, -0.041615907, 0.017484175, -0.035297982, -0.0049719336, 0.06378359, -0.03683626, -0.012065867, 0.018706556, -0.021137586, 0.00468694, -0.034116805, -0.05279589, 0.023692226, -0.022359967, -0.071859546, -0.0039658723, -0.048345875, 0.0048242863, 0.037385646, -0.032358773, 0.04636809, 0.0013322585, -0.056311954, 0.0780676, -0.008563537, 0.067574345, -0.012951749, 0.025024483, -0.040654484, -0.006935985, 0.03213902, -0.037001073, 0.0031194764, 0.026645169, -0.02138481, 0.019091126, -0.0051951213, 0.03205661, -0.013466798, 0.024639914, -0.03175445, 0.0023263018, -0.0019623344, 0.033759706, -0.01990147, 0.02753792, -0.048263468, 0.015781082, -0.013137167, 0.01547892, 0.010733608, -0.00073480234, 0.05911382, 0.0073411562, 0.0038147916, 0.007313687, -0.00848113, 0.001022371, -0.023596082, 0.0404622, -0.016234325, 0.05427923, -0.003591604, 0.0040654484, -0.013391257, 0.03502329, -0.030078825, 0.039308492, -0.017456707, 0.04916995, 0.050735697, 0.002364072, -0.0005347919, 0.030820495, 0.03650663, -0.038456947, 0.06488236, -0.010850352, -0.021755643, -0.011070106, 0.023101637, -0.021700704, 0.019970141, 0.022703333, 0.008542935, 0.02569748, 0.018651618, 7.2053124e-05, 0.0024121432, 0.00018649043, 0.014187866, 0.017484175, -0.0077600623, -0.044719934, -0.031205064, -0.017717663, -0.037193358, 0.042412516, -0.030326048, 0.0027503583, 0.03672638, -0.03315538, -0.024667384, 0.02192046, -0.014709782, -0.03672638, 0.0148333935, 0.019928938, 0.015286636, 0.012690792, 0.008604742, 0.028540548, 0.014517496, 0.03279828, -0.034885943, 0.01121432, 0.004261167, -0.0028877046, -0.021590827, 0.028458139, -0.012979219, -0.019324616, -0.0071694735, -0.029254748, 0.00030688298, -0.0020292907, -0.018431865, 0.006242386, -0.017030932, -0.0284032, -0.012155142, -0.02712588, 0.01969545, 0.0064278035, 0.023225248, 0.0029941478, -0.013041025, 0.0184456, -0.020450855, 0.021233728, -0.015011943, 0.027469246, -0.018582946, -0.009868327, 0.0111319125, 0.025903499, -0.015877225, -0.015025678, -0.03205661, -0.033732235, 0.01948943, -0.014819658, -0.018582946, 0.011049504, 0.008694016, -0.014393886, 0.009840857, 0.018994983, -0.03620447, 0.032523587, 0.05172459, -0.016028306, 0.029199809, 0.01606951, -0.012436701, -0.022236355, 0.002631897, 0.0059436583, 0.001731421, -0.015066882, 0.021192525, 0.0057754093, 0.021288667, -0.05867431, 0.01734683, 0.01075421, -0.0041753254, 0.00014303322, -0.028513078, -0.053427685, 0.0025803924, -0.0016181103, 0.026301803, -0.0012661606, 0.01642661, 0.015726143, -0.028650424, 0.01323331, -0.002206124, -0.00756091, -0.0037152155, 0.0009562732, 0.018047296, -0.00986146, -0.02646662, 0.0043435744, -0.0015288353, -0.03494088, 0.037303235, 0.009470022, 0.014668577, -0.016330468, 0.030051356, 0.008316315, 0.00720381, 0.009284605, -0.00955243, 9.598141e-05, 0.005538487, -0.010067479, 0.0072244117, -0.03713842, -0.0032259196, -0.0019692017, 0.016124448, -0.022002866, -0.026631435, 0.006812373, -0.027661532, 0.063399024, 0.032990564, 0.0027022872, -0.029117402, 0.0038731638, -0.025889765, 0.007911143, 0.013803297, -0.016866118, 0.032386243, -0.028059835, -0.014654843, 0.010362773, 0.02278574, -0.006383166, 0.011379136, -0.0028173146, 6.2556916e-05, -0.04340141, 0.009689777, 0.027881285, 0.015781082, -0.013830766, 0.042412516, -0.014064254, -0.0213024, -0.02848561, 0.013295115, -0.0039143674, 0.031479757, -0.053427685, -0.0070870654, -0.013404992, -0.016742505, 0.013075361, -0.031946734, -0.0020893796, 0.007938612, -0.010246029, -0.018788965, -0.0033357965, -0.015053147, -0.032743342, 6.1001043e-05, 0.00042191046, 0.03941837, -0.013006688, -0.05845456, 0.012319957, -0.008412457, 0.019118596, -0.01054819, -0.023101637, 0.0078493375, 0.007869939, 0.037742745, 0.014792189, -0.053812254, -0.026919862, 0.0146823125, -0.0020807954, 0.018679088, -0.02950197, 0.036149528, -0.02418667, 0.019970141, -0.017223218, -0.048730444, -0.0139749795, 0.022936821, -0.038127314, -0.027592858, 0.0073686256, -0.009332676, -0.012924281, -0.0180885, -0.026425416, -0.0015957915, -0.008213305, 0.010630598, -0.0020842291, -0.05579004, -0.0072793504, 0.029804133, -0.0035400991, 0.010197958, -0.017236952, -0.007272483, -0.002065344, -0.00656515, -0.006455273, 0.018170906, 0.0048242863, 0.0136728175, -0.008014153, 0.018019825, -0.048290934, -0.034089334, 0.0032739907, 0.011942255, -0.0018816434, -0.01660516, 0.02150842, 0.027139615, -0.0075059715, -0.013494267, 0.005857817, -0.0030490863, 0.019956408, -0.017401768, 0.023101637, 0.0316171, -0.019008718, -0.009421951, 0.018555475, 0.006870745, 0.003311761, 0.023156574, 0.0023606385, 0.004896393, 0.033979457, 0.0101224175, 0.011379136, -0.0205882, 0.036341812, -0.009840857, -0.029804133, 0.011200585, -0.010685537, -0.023156574, -0.02439269, 0.0042336974, -0.023266451, 0.009854592, -0.026809985, -0.036946137, -0.023266451, -0.020945301, -0.0064106355, 0.017388033, 0.02810104, -0.025271706, 0.0018061029, 0.011557685, -0.006935985, -0.12943509, -0.00018112535, 0.013089096, -0.011104443, -0.008233907, -0.013075361, 0.01356294, 0.0063316613, -0.0015399946, -0.013528604, 0.027551655, -0.016440343, 0.016509017, -0.007574645, 0.017195748, -0.012889944, 0.007856204, -0.02109638, 0.02240117, -0.02950197, -0.042110354, 0.023376329, -0.025930969, 0.010438314, -0.045379195, 0.026068315, -0.03315538, 0.006702496, 0.016124448, 0.036122058, -0.004240565, 0.0019468828, -0.0052328915, 0.015781082, -0.026494088, -0.003917801, -0.004498089, 0.0014361265, 0.0003573148, 0.0063797324, 0.058124926, -0.029419564, 0.019942673, -0.03972053, 0.019036189, -0.020931566, -0.026301803, -0.008748955, 0.014517496, 0.008886301, 0.008783292, 0.0036087723, 0.0073686256, 0.012052132, 0.018129703, -0.029749194, 0.00031847158, -0.019983876, 0.013061627, 0.032276366, 0.026809985, 0.04329153, 0.017786337, -0.015327839, 0.013315718, -0.0066475575, 0.012471038, 0.01747044, -0.02183805, -0.03458378, -0.025862295, 0.025477726, -0.024873402, -0.010623731, 0.0142016, 0.028732833, 0.046175804, 0.016412875, 0.0019382987, -0.0045736297, 0.06202556, -0.023870775, 0.0001167443, 0.01856921, -0.00015505102, 0.04532426, 0.004738445, 0.0042817686, 0.026562762, 0.029859072, 0.039528247, -0.029694255, 0.003303177, 0.01714081, -0.048620567, -0.0029254747, 0.027098412, 0.015602532, 0.03749552, -0.0003764145, 0.028595485, -0.0063866, -0.005696435, -0.004450018, -0.03364983, -0.011502747, -0.007492237, -0.02848561, -0.022181416, 0.017690195, 0.03246865, 0.041039053, 0.0047830828, 0.03312791, -0.0043367073, 0.011482145, 0.032111548, 0.002575242, -0.0102734985, -0.027661532, 0.00010590682, 0.025203034, -0.02064314, 0.0059367907, 0.022346232, 0.049664397, 0.0018352891, -0.009009913, 0.008790159, 0.02373343, -0.0012661606, -0.011090708, -0.038484413, -0.0006657, -0.0154239815, -0.0051264483, -0.009627971, -0.010108682, 0.008790159, 0.027084677, -0.040022694, 0.0053427685, 0.02242864, -0.014242805, -0.058124926, -0.026219396, -0.015259166, -0.029282216, -0.022978025, 0.027441777, 0.023609817, 0.026370477, -0.0072656157, 0.01246417, -0.0066784606, 0.03793503, 0.03104025, -0.030875433, -0.027963692, 0.042742148, 0.023568613, 0.030326048, 0.025065687, -0.008865699, 0.008288845, 0.01565747, -0.020533262, 0.004412248, -0.020656873, -0.0059608263, -0.02848561, -0.0052122897, 0.014696047, 0.018802699, 0.01877523, 0.020849159, -0.003220769, 0.020849159, -0.016151916, 0.027373103, -0.009435686, 0.014888331, -0.0026198793, -0.011798041, 0.01021856, -0.018431865, 0.018582946, 0.020505793, 0.021274932, -0.032990564, 0.012835005, 0.037770215, -0.003093724, 0.025601339, -0.024543772, -0.0034044697, 0.016838647, 0.025367849, -0.034446437, -0.022140212, -0.050598353, 0.049115013, 0.000785878, -0.019667981, 0.043483816, 0.025051953, -0.011791174, -0.0023503373, 0.0075471755, 0.023582349, -0.015602532, -0.0018387227, -0.00028563724, -0.005408008, -0.031205064, -0.0030576705, -0.015163024, -0.007451033, 0.013109698, -0.032578528, -0.038923923, -0.034006927, 0.0021803714, 0.027194554, 0.023211513, 0.019709185, 0.008089693, 0.004401947, 0.020093754, -0.006877613, -0.04477487, -0.010149887, -0.019819062, 0.013995581, -0.025271706, 0.04669772, -0.00883823, -0.0043710438, -0.011179984, 0.025354115, 0.046093397, -0.0138238985, -0.0422477, -0.031672042, -0.016412875, -0.015039412, 0.024832198, 0.03568255, 0.009648573, -0.030463394, -0.035078228, 0.010314702, 0.0060913055, -0.016302997, 0.023156574, 0.005366804, 0.04447271, 0.004669772, 0.05026872, 0.024571242, -0.013109698, 0.019049922, -0.009765317, -0.04345635, 0.013535472, 0.011543951, -0.021879256, -0.012711394, 0.0379625, 0.0033426639, 0.00848113, 0.033594888, -0.0012455586, 0.021769378, -0.0070801983, -0.029584378, 0.009792786, -0.00033070397, 0.022936821, -0.023719694, 0.018486803, 0.0005738497, 0.0012841873, 0.0023022662, -0.00020129807, 0.0015279768, -0.005909322, -0.0009888929, -0.002312567, -0.0068673114, -0.019709185, 0.033704765, 0.013013556, -0.017030932, -0.025326645, 0.013343186, -0.018129703, 0.023225248, 0.0026799683, -0.038841516, -0.0017700496, -0.017813805, 0.025573868, -0.016687566, -0.020546997, 0.013459931, 0.028595485, -0.055542815, -0.031507228, 0.025958437, -0.020134958, 0.02388451, 0.006554849, 0.03282575, -0.0078081335, -0.02477726, 0.008419324, -0.035435326, 0.01580855, -0.030738087, -0.009923265, 0.04584617, 0.011386002, 0.031287473, -0.016509017, -0.017456707, 0.0109121585, 0.009895796, -0.0049684998, 0.011207453, 0.009916398, 0.015616267, -0.016495282, 0.04480234, -0.017800072, 0.031287473, -0.007313687, -0.012958617, 0.019461961, -0.023829572, -0.017923683, 0.006465574, -0.025299177, -0.009964469, 0.020148693, -0.020656873, 0.0017803506, -0.0087420875, -0.0016035172, 0.018706556, 0.083231814, -0.057355788, -0.023939447, -0.028952586, -0.0014180999, -0.0049204286, -0.016728772, -0.017305626, -0.05724591, -0.021426013, -0.014270274, 0.04233011, 0.0064003346, 0.008075958, 0.0068604443, 1.4365021e-05, -0.018788965, 0.009662308, 0.028238386, 0.009209065, 0.018390661, 0.019544369, -0.03749552, -0.0027692434, 0.006723098, 0.023390064, 0.018720292, 0.007897409, 0.01856921, 0.025326645, -0.022634659, -0.054883555, 0.011296728, -0.0129654845, -0.0129998205, 0.00030302012, 0.00904425, 0.0065926192, -0.015053147, -0.004559895, -0.016138183, -0.008233907, -0.002312567, -0.0070870654, -0.009724113, 0.0048105516, 0.00468694, 0.004515257, 0.005497283, -0.032633465, 0.01111131, -0.005638063, -0.013178371, -0.018061029, 0.03818225, -0.032880686, -0.004092918, 0.04518691, -0.020244835, 0.0037220828, -0.01731936, 0.04669772, -0.016412875, -0.0054560793, -0.010781679, -0.0422477, -0.02334886, 0.019750388, -0.0050165714, 0.027606593, -0.04730204, -0.0042714677, 0.027963692, 0.003821659, 0.012539711, -0.008831363, -0.027579123, -0.021851785, 0.042522393, -0.027235758, -0.005802878, -0.054471515, -0.02914487, -0.002465365, -0.020698078, 0.004089484, 0.030655678, 0.012786934, -0.050790638, -0.0065308134, -0.009181595, 0.013988714, 0.017717663, 0.040379792, 0.008687149, -0.008694016, -0.020656873, -0.021206258, 0.0051813866, 0.0013837633, 0.0014799057, -0.007643318, -0.02064314, 0.007890541, -0.006352263, -0.003701481, -0.034446437, 0.003629374, 0.003660277, 0.0027984295, -0.0043641767, 0.010438314, -0.008343783, 0.011173116, -0.008130897, 0.031177595, 0.01714081, 0.055707633, -0.025848562, -0.032935627, 0.013006688, -0.01494327, -0.011530216, -0.021426013, -0.0023675056, -7.602329e-05, 0.010369641, 0.01586349, -0.005737639, 0.0077119907, -0.0013777544, -0.011056372, 0.02641168, -0.016275529, -0.020739282, 0.0020224233, 0.0057342052, 0.010561925, -0.044005733, 0.05103786, -0.013679685, 0.005226024, -0.039912816, -0.05296071, -0.0004613975, 0.038868982, 0.011351666, 0.014029917, -0.003983041, -0.022565985, -0.03285322, -0.018582946, 0.026329273, -0.007828735, 0.021398542, -0.040269915, -0.0051916875, 0.010342171, 0.024681117, 0.007155739, 0.020093754, 0.010809149, -0.02171444, -0.008728353, -0.0030576705, -0.02260719, -0.03095784, 0.019667981, 0.04559895, -0.00022297303, -0.023472471, -0.01762152, -0.0013614445, -0.009215932, -0.010342171, 0.0073480234, 0.0003146087, 0.014641108, -0.014036785, 0.024763526, -0.0072312793, -0.014270274, 0.014476293, -0.017964887, -0.012498507, 0.023060432, -0.0069016484, -0.008508599, -0.0061428105, 0.006723098, 0.011866715, -0.024997015, -0.002743491, 0.009421951, 0.028760301, 0.04233011, -0.002444763, 0.023101637, 0.023664756, -0.009305207, 0.01221008, -0.0128144035, -0.0036431088, -0.02370596, -0.026727578, 0.03104025, -0.0006373724, -0.022126477, -0.006503344, 0.02005255, -0.026397945, 0.00055024336, -0.0033460977, -0.025958437, 0.004302371, 0.00401051, 0.011351666, -0.015094351, 0.003519497, 0.0009185029, -0.025985908, 0.020862892, 0.030573271, -0.0044225487, -0.01749791, 0.016454078, -0.030106295, -0.025972173, -0.017511645, -0.019915204, 0.02038218, -0.036369283, -0.03216649, 0.012532843, 0.0102734985, 0.008639078, -0.025354115, 0.0022971157, 0.0077463277, 0.03062821, 0.019750388, 0.014077988, -0.010472651, 0.021673236, -0.014174131, 0.027689, 0.01494327, -0.019173535, -0.026645169, -0.009518094, 0.020945301, -0.005806312, -0.014888331, -0.005208856, -0.025436522, -0.041423623, 0.010671802, -0.0027778274, -0.021467216, -0.00053736713, -0.07938612, -0.011509614, -0.0033855846, 0.012203213, -0.0058269138, 0.04732951, -0.0008202145, -0.012457304, -0.023431268, 0.016124448, -0.0053977068, 0.0061805802, -0.010994566, -0.021467216, 0.028925117, 0.06367371, 0.005905888, -0.0052397586, -0.016536487, 0.0015348442, -0.010486385, -0.0009949018, 0.004796817, -0.017401768, -0.017278155, 0.019420758, 0.028004896, 0.0082545085, 0.029639317, -0.010987699, 0.009140392, 0.04005016, -0.0063213604, -0.006074137, 0.0080690915, 0.0163442, 0.012910546, -0.031205064, 0.0026559327, 0.010493252, -0.038429476, -0.015080616, -0.00029465058, 0.0010292383, -0.0028756869, 0.05147737, -0.014462559, 0.02010749, -0.01203153, -0.021398542, 0.0002957236, -0.005754807, -0.0109121585, 0.0070870654, 0.0180885, 0.011042637, 0.025917234, -0.016481549, -0.0057822764, 0.0036156396, -0.00055753987, 0.016385406, 0.011612624, -0.019832795, 0.0059814285, -0.005435477, -0.0063282275, 0.044719934, 0.0039967755, -0.023980653, -0.014064254, 0.019242207, 0.027043473, 0.0087077515, -0.0033272125, 0.0034164875, -0.0034645586, 0.00045109654, 0.016357936, -0.028513078, -0.0047727814, 0.0071694735, 0.0036877464, -0.013137167, 0.025216768, 0.015547593, -0.0036156396, 0.024845934, -0.016715037, 0.016509017, -0.0022421773, 0.0032945927, -0.029996417, 0.0021288665, 0.014064254, -0.032743342, -0.012285621, -0.005469814, 0.019599307, -0.04661531, -0.044939686, -0.005995163, 0.00026288928, 0.027070943, -0.0058200466, 0.013425594, -0.011550819, -0.062520005, -0.037825152, -0.0012781784, -0.0052534933, 0.002764093, 0.02988654, -0.010108682, -0.015355309, -0.022717066, 0.013961244, -0.017566584, 0.01874776, -0.014215335, 0.049087543, -0.02109638, 0.0138238985, 0.000725789, 0.024008121, -0.01790995, -0.005198555, 0.029666787, -0.0019932373], "content_hash": "f2efecf3a4321698359e809d348f9fcd"}, {"content": "text. ### Generation Changed The head was important enough that removing it altered the model's entire output sequence. This is a strong signal of importance. Look at where the texts diverge -- the point of divergence tells you where the head's contribution was most critical. ### Generation Stayed the Same\n\nEven if the probability shifted, the model still chose the same tokens. This means the head's contribution wasn't large enough to cross the decision boundary. The model is robust to losing this head for this particular input.\n\n## Multi-Head Ablation\n\nWhen you ablate multiple heads simultaneously:\n\n- **Additive effects**: If ablating heads A and B together has a bigger effect than either alone, the heads contributed independently to the prediction.\n- **Redundant heads**: If ablating both has about the same effect as ablating just one, the heads may have been providing the same information.\n- **Synergistic effects**: Rarely, ablating two heads together can have a much larger effect than the sum of their individual effects. This suggests the heads work together as a circuit.\n\n## Tips for Interpretation\n\n- Always compare ablation effects across different head categories on the same prompt\n- Try the same head on multiple prompts to see if its importance is consistent or input-dependent\n- A head's category (Previous-Token, Syntactic, etc.) gives you a hypothesis about why it matters -- ablation lets you test that hypothesis\n- Remember that ablation is a blunt tool: removing a head removes all of its functions, not just the one you're interested in", "source_file": "interpreting_ablation_results.md", "chunk_index": 1, "embedding": [0.055395074, 0.031762846, 0.065832645, 0.017442837, -0.0125194555, -0.0041215736, -0.007821144, 0.0053348355, 0.00547902, 0.03719263, 0.05111877, -0.033113256, -0.059193112, 0.030721901, 0.002945237, 0.0059221247, 0.014531009, -0.013771402, -0.003812104, 0.05303185, 0.0016431786, 0.0053629694, 0.024096435, 0.02954029, 0.04124387, -0.020270266, 0.029287087, 0.028977618, 0.052834917, -0.016261226, 0.009783463, -0.017259968, -0.01676763, -0.011766882, -0.061950207, 0.03032803, 0.021184606, 0.015164015, 0.007378039, 0.005486054, 0.046504855, -0.044704303, -0.059305646, 0.047067527, -0.02606579, 0.036376756, 0.047123794, -0.03477314, -0.020987673, 0.074160196, 0.0006461938, 0.06256915, -0.013722167, 0.027092665, -0.040512398, -0.0009486301, -0.014369241, -0.01862445, 0.013672934, -0.029512156, 0.0024704826, 0.009692028, 0.026361192, 0.020495335, -0.0007420239, -0.033225793, -0.014742011, 0.0060557593, -0.019918595, -0.0067590997, 0.025545316, -0.017302169, 0.013025861, 0.002783469, -0.031228306, -0.028977618, 0.020875137, 0.008489316, -0.027444335, -0.015304683, 0.05885551, 0.031228306, 0.057195626, -0.044563636, -0.025700051, -0.03632049, -0.039161984, -0.017274035, -0.026220523, -0.05342572, -0.034519937, 0.01664103, 0.0131946625, -0.0036503358, -0.026881663, 0.020495335, -0.0049233814, -0.022971092, -0.01593769, 0.013651834, 0.0027869856, 0.0016546078, -0.03288819, -0.0021451877, 0.015220283, -0.031622175, -0.036376756, 0.005014816, -0.03508261, 0.0028661115, -0.019355923, -0.03336646, -0.031678442, 0.022169285, -0.008383815, -0.04692686, -0.04189094, 0.035982884, 0.003977389, -0.02245062, -0.00095830107, -0.013926136, -0.00478623, 0.0004048602, -0.018272778, -0.011091676, 0.033478994, -0.01050087, -0.013447865, -0.016303426, 0.028808815, 0.028161742, -0.027050465, -0.03508261, -0.0653825, 0.0010550104, 0.0016774664, 0.031200172, -0.060937397, 0.029427756, 0.008025112, -0.013581499, -0.03648929, 0.0072162705, -0.059924588, 0.020213997, 0.009952264, 0.059361916, -0.009481026, 0.046983127, -0.010177333, 0.004234108, -0.0049515152, -0.020087397, -0.0070967027, -0.04366336, -0.024659108, 0.010655604, -0.0027711606, -0.062794216, 0.025390582, 0.022295885, -0.04234108, 0.00087214186, -0.03547648, 0.038599312, -0.021578478, 0.00044969816, -0.02931522, -0.03522328, -0.04085, -0.0442823, -0.030524965, -0.028597813, -0.02280229, 0.0059748753, -0.070334025, -0.021817613, 0.0101703, -0.043213222, 0.016964566, 0.044535503, -0.024926377, 0.035814084, -0.025278047, -0.041694008, -0.007905544, 0.021902014, -0.04523884, 0.00477568, 0.029343354, 0.0033672412, 0.04793967, 0.002162771, -0.0045154444, -0.035842218, 0.06594518, 0.012955527, -0.002076612, 0.005658372, 0.032325517, 0.013475998, 0.010634504, -0.0028063275, -0.016036157, -0.0049128314, -0.011844249, 0.021733213, -0.040484264, -0.034266736, 0.016387828, -0.064144626, 0.003921122, 0.039865322, -0.030440565, -0.027697537, -0.040878132, 0.03271939, 0.0127234245, 0.010704838, 0.020987673, -0.010613404, 0.03857118, 0.045013774, -0.040371727, 0.01329313, 0.059361916, -0.008362715, 0.00880582, -0.037080098, 0.04045613, 0.023491563, 0.008243147, -0.0057709063, -0.034913808, -0.077536225, 0.029877892, -0.0020432032, 0.032860056, -0.030299896, -0.019960795, -0.007518707, 0.03632049, 0.0238151, -0.006541064, 0.04824914, -0.027458401, -0.05269425, 0.006636015, 0.04701126, 0.058180302, 0.0006259728, -0.050415426, 0.025601584, -0.031200172, 0.003089422, 0.0048811813, 0.025700051, -0.009945231, 0.03032803, -0.045632713, -0.0008739002, -0.03260685, 0.028724415, -0.033000723, -0.0072514378, 0.0041321237, 0.005556388, -0.048614874, -0.029849758, -0.055845212, -0.019749794, 0.014432541, 0.00781411, -0.047883403, -0.010634504, -0.02429337, 0.0060663093, -0.05109063, -0.0036081353, -0.0019922112, -0.022816356, 0.05286305, -0.018849518, -0.019904528, 0.0134197315, -0.023632232, 0.016415961, -0.0011561154, -0.0027113766, 0.015079615, -0.039696522, 0.080574654, -0.04062493, -0.011429278, 0.030215496, -0.0049831653, 0.04287562, -0.028794749, 0.0034621921, 0.031312708, -0.020270266, 0.015248416, -0.0056970557, 0.024082368, -0.0021240874, 0.00882692, 0.010381302, -0.022253685, 0.012132619, -0.018216511, 0.0021152957, -0.033563394, -0.034801275, -0.06476357, -0.032184847, 0.031734712, -0.03539208, 0.0066922824, -0.014573209, -0.008418983, -0.034351137, 0.004399393, -0.0025355415, -0.0014260223, 0.044394836, -0.0114363115, 0.0015957031, -0.008440083, 0.004198941, 0.034829408, 0.00072927587, 0.013532266, 0.029062018, -0.03755837, -0.0077015753, -0.053003717, 0.05145637, 0.013222796, -0.0023438812, -0.05789897, 0.007001752, -0.018933918, 0.028147675, -0.0061331266, 0.020410933, -0.0026164255, 0.026614394, 0.03437927, 0.0069138343, -0.012484289, -0.00375232, -0.007117803, -0.029821625, 0.02755687, 0.08158746, -0.02015773, -0.016415961, 0.03446367, 0.0015455901, 0.010747039, 0.017147435, -0.0046244618, 0.02086107, 0.012041184, -0.01395427, -0.047911536, 0.038177308, -0.0057779397, 0.016922366, -0.01936999, -0.0034903258, -0.01333533, -0.0009846763, -0.10173113, -0.008665152, -0.051484503, -0.0017768132, -0.032184847, 0.033028856, -0.011443345, -0.024068302, -0.045717113, -0.05286305, 0.033394594, 0.013483032, -0.003836721, 0.012716391, -0.06408836, -4.060691e-05, 0.030693768, -0.004343126, 0.002141671, 0.030440565, -0.0064672134, 0.057336293, -0.038683712, 0.040118527, 0.035645284, 0.050556093, 0.02600952, 0.014559142, -0.054691736, -0.010690771, 0.043353893, 0.04802407, 0.003590552, -0.012041184, 0.030862568, 0.026445592, 0.070502825, 0.035335813, 0.032466184, 0.014144172, 0.030131094, -0.0011060025, 0.038599312, 0.041525207, 0.041750275, 0.005953775, -0.014826411, -0.08316295, -0.012385821, -0.026417458, -0.03001856, -0.028147675, 0.010606371, 0.008629984, 0.036207955, -0.044310432, -0.05260985, 0.016697297, 0.0442823, -0.028555613, -0.0038965049, -0.022084882, 0.0027359934, 0.062794216, -0.044169765, -0.034576204, 0.03421047, -0.011816116, -0.003977389, -0.037670903, -0.060149655, 0.013257963, 0.012202953, -0.05846164, 0.001881435, -0.059586983, -0.01672543, 0.005282085, -0.0074905735, 0.04917755, 0.0146857435, -0.031987913, 0.07573567, -0.0006989443, 0.055704545, -0.021254942, 0.02522178, -0.030750034, -0.0039246385, 0.013651834, -0.017639773, 0.022366218, 0.026909797, -0.01223812, 0.039077584, -0.004009039, 0.012153719, 0.011112776, -0.005612655, -0.029343354, 0.019932661, -0.027697537, 0.012097452, -0.008629984, 0.039274517, -0.04326949, 0.011795015, -0.02579852, 0.029849758, 0.0052082343, -0.028836949, 0.068027064, 0.015262483, 0.0036538525, 0.026164256, 0.010465702, 0.016387828, -0.019862328, -0.0010576479, -0.0168661, 0.057673898, 0.0062491777, 0.024405906, -0.005813107, 0.018765116, -0.03010296, 0.07219084, 0.0055352873, 0.004177841, 0.017105235, -0.007202204, 0.00033320743, 0.04476057, 0.032550585, -0.028766615, 0.039640255, -0.01918712, -0.033169523, -0.0038859546, 0.013321264, -0.02443404, 0.022703823, 0.011373011, 0.013075095, 0.02698013, 0.017822642, -0.0034498838, 0.0114363115, -0.01085254, 0.012076352, 0.0010752314, -0.02596732, -0.041581474, -0.006505897, -0.027134866, -0.021058006, 0.026375258, -0.02786634, 0.0035501097, 0.029906025, -0.030046694, -0.024701308, 0.015304683, -0.0052856016, -0.029877892, 0.017105235, 0.0050640497, 0.002222555, 0.0036433022, -0.017527238, 0.015149948, 0.002823911, 0.045098174, -0.03345086, 0.01906052, 0.0067590997, -0.0063968794, -0.0062491777, 0.02346343, -0.023941701, -0.028499346, -0.013553366, -0.020804804, -0.00089368166, 0.0035378013, -0.022703823, 0.023308694, -0.02681133, -0.009762363, -0.01047977, -0.037136365, 0.017020833, -0.00078730145, 0.050949965, 0.023829166, -0.019215254, -0.002542575, -0.026951997, 0.026797263, -0.018202445, 0.0025619168, 0.0069314176, 0.0011921617, 0.005563421, 0.022928892, -0.0077859764, -0.012202953, -0.023294628, -0.026037656, 0.015445352, -0.025292113, -0.024645042, 0.02346343, 0.020523468, -0.050499827, 0.018230578, 0.025812587, -0.030918837, 0.04543578, 0.039949723, -0.01773824, 0.016176825, 0.027022332, -0.0018427514, -0.010669671, -0.006214011, 0.02143781, 0.0030612883, -0.009445859, 0.018779185, -0.0055106706, 0.0076171746, -0.0373333, 0.018469714, 0.012146685, 0.00644963, 0.018469714, -0.009706095, -0.040343594, 0.008665152, -0.019482523, 0.018511914, 0.019609125, 0.00023935548, 0.03615169, -0.031059504, 0.003024363, -0.009150457, 0.011154976, -0.0030226046, 0.0069490015, 0.029793492, -0.0005758598, -0.028021075, 0.028274277, 0.012786726, -0.045379512, 0.035167012, 0.027444335, 0.013279064, -0.013806568, 0.009980397, 0.008039178, -0.010550103, 0.0074061723, -0.02319616, 0.0014312974, -0.0015719654, 0.0052856016, -0.0010523729, -0.018371247, 0.016401894, 0.0061120265, 0.017288102, -0.021606611, -0.02654406, 0.0140527375, -0.039190117, 0.06403209, 0.03032803, 0.019173054, -0.0203828, -0.011316744, -0.020987673, 0.010247667, 0.007117803, -0.017977376, 0.009115289, -0.025911054, -0.044788703, 0.00608741, 0.0105571365, -0.008411949, 0.021676946, -0.0050675664, -0.009649828, -0.045857783, 0.016486295, 0.051175036, 0.018905785, -0.004156741, 0.04374776, 0.007117803, -0.009663895, -0.028147675, 0.0060065254, -0.046110984, 0.03997786, -0.036995694, -0.017639773, -0.022942958, -0.023702566, 0.0065375473, -0.03353526, -0.007378039, 0.013862835, -0.0019078103, -0.037952237, 0.019158987, -0.0067837164, 0.008313482, -0.006650082, -0.013658867, 0.048643008, -0.015600086, -0.041018803, -0.0027957773, -0.025573451, 0.030215496, 0.007518707, -0.018652583, 0.0055317706, 0.010275801, 0.05902431, 0.023336828, -0.034519937, -0.023829166, 0.00712132, 0.01976386, 0.015811088, -0.008707352, 0.031397108, -0.022127083, -0.007574974, -0.004174324, -0.04374776, -0.025854787, 0.010972108, -0.007849277, 0.0034534005, 0.0038191373, -0.01756944, 0.0101491995, -0.0442823, -0.020847004, -0.0044662105, -0.0033619662, 0.015951756, 0.008102479, -0.03587035, 0.0018656099, 0.025657851, -0.014756078, -0.0038929882, -0.008693285, -0.0059115747, 0.01575482, -0.03305699, 0.011555879, 0.022942958, 0.0123154875, 0.012357688, -0.001244033, 0.031087637, -0.02003113, -0.0045365444, -0.004800297, 0.013222796, -0.0018779184, -0.0003191406, 0.008756585, 0.03992159, -0.014003504, 0.009396626, 0.013595566, 0.03271939, 0.016711364, -0.023210227, 0.008960554, 0.008123579, -0.017091168, -0.020748537, -0.0046033617, 0.00025430144, -0.005201201, 0.021226808, 0.002171563, 0.02553125, 0.034829408, 0.0042305915, 0.02094547, -0.03719263, 0.049290083, -0.025292113, -0.03226925, 0.01804771, 0.029062018, -0.0072162705, -0.047911536, 0.014087904, -0.015698554, 0.025742251, -0.024419973, -0.024841977, -0.015515685, -0.04296002, 0.016261226, 0.018033642, 0.022506887, -0.014868612, -0.003270532, 0.021254942, -0.0144606745, -0.13549146, -0.0038754046, 0.018497847, -0.036686227, -0.029793492, -0.0021240874, 0.028738482, 0.027050465, -0.007279571, -0.0077930097, -0.000348593, 0.006425013, 0.0015693279, -0.011872383, 0.026051722, -0.056632955, -0.009945231, -0.03446367, 0.031059504, -0.045210708, -0.033957265, 0.0045576445, -0.026572194, 0.00041760824, -0.05111877, 0.0021645294, -0.019482523, -0.0017108751, -0.019524725, 0.027697537, -0.027317734, -0.0062315944, -0.028133608, 0.027064532, -0.021803547, 0.016936433, -0.0025320249, -0.013518199, 0.003611652, -0.00050684455, 0.020298399, -0.03103137, -0.002823911, -0.022971092, -0.009150457, -0.00475458, 0.0007143299, -0.00237729, 0.009094189, 0.00038200166, -0.006164777, -0.006836467, -0.006027626, 0.028921349, 0.014516942, -0.03966839, -0.012055252, 0.013082128, 0.017935175, 0.02284449, 0.02271789, 0.045126308, 0.0062386277, -0.0046666623, 0.0074202395, -0.00479678, -0.010233601, 0.037164498, -0.035420213, -0.03376033, -0.01932779, 0.023843233, -0.016387828, 0.010810339, 0.02579852, 0.020185864, 0.028541546, 0.034238603, -0.009431792, 0.009382558, 0.059305646, -0.033788465, -0.007188137, 0.0238151, -0.024110503, 0.05083743, -0.0042763087, 0.012786726, 0.01936999, 0.04529511, 0.025629718, -0.048389807, -0.0034358168, 0.03705196, -0.05159704, -0.017878909, 0.012006017, 0.0063019283, 0.023843233, -0.005633755, 0.017133368, -0.008362715, -0.003801554, -0.047602065, -0.03511074, -0.0129695935, -0.002111779, -0.002823911, -0.01432704, -0.0011613906, 0.022324018, 0.020453133, 0.015262483, 0.03150964, 0.0073850723, 0.025109246, 0.027570937, -0.0024792743, -0.028119542, -0.028640013, -0.0051343837, 0.037023827, -0.016922366, -0.012139652, 0.0064390795, 0.029906025, -0.0140527375, -0.0008615918, 0.01761164, 0.02152221, -0.0051238337, -0.023224294, -0.03679876, 0.009804563, -0.0092911245, -0.016345626, -0.011773915, -6.2146704e-05, -0.0033461412, 0.014130105, -0.025714118, -0.021226808, 0.030721901, -0.029962294, -0.053903993, -0.05497307, -0.006579748, -0.011942717, -0.04718006, 0.0045119277, 0.016894232, 0.034323003, -0.00019495712, 0.004670179, 0.0011728198, 0.0078070764, 0.02848528, -0.021142406, -0.015965823, 0.04552018, 0.013159496, 0.02288669, 0.027247401, 0.003360208, 0.018919852, 0.008756585, -0.0030331546, 0.013370497, -0.01901832, -0.0028080859, -0.02249282, -0.01734437, -0.008531516, 0.0124983555, 0.015445352, 0.020790737, -0.009748296, -0.003216023, -0.011302677, 0.026529994, 0.011872383, 0.0046736957, -0.019918595, -0.023857301, 0.008580751, 0.0040864064, 0.02923082, 0.024602842, 0.03843051, -0.019496592, 0.0081657795, 0.03547648, -0.0025513666, 0.016753564, -0.020565668, -0.011956784, 0.024448106, 0.020059263, -0.031200172, -0.036545556, -0.06735186, 0.05168144, 0.010085898, -0.027064532, 0.0056513385, 0.036939427, -0.02782414, 0.0017047209, -0.00094687176, 0.0077789426, -0.009537294, 0.020804804, 0.011696548, -0.0012950252, -0.03494194, -0.009762363, -0.012301421, 0.013630734, 0.030356163, -0.021972349, -0.037839703, -0.026600327, 0.0062210443, 0.026093923, 0.00050640496, 0.008644051, 0.017485037, 0.00648128, 0.008411949, 0.0024265237, -0.03857118, -0.037755303, -0.019116787, 0.017499104, -0.015515685, 0.02615019, -0.030975103, -0.003910572, -0.008700319, 0.027374001, 0.04523884, -0.007891477, -0.03159404, -0.028893216, -0.00542627, -0.03508261, 0.017921109, 0.017259968, 0.004729963, -0.01941219, -0.047208194, 0.0011552363, 0.029793492, -0.022549087, 0.029652823, 0.0003804631, 0.03305699, 0.022338085, 0.04296002, 0.01567042, 0.0016396618, 0.020087397, 0.00019572639, -0.03648929, 0.013799535, 0.0050534992, -0.036433022, 0.0020432032, 0.02526398, -0.018427514, 0.003847271, 0.04476057, -0.010233601, 0.0306375, -0.010536036, -0.016626963, 0.014671677, 0.0015482276, -0.004483794, 0.0021803547, 0.036123555, 0.017414704, -0.0065480974, 0.0052539515, -0.00955136, -0.017752307, -0.0058201402, 0.009445859, 0.00957246, -0.021339342, -0.018779185, 0.0041426737, 0.009720162, -0.00044200537, -0.004096957, 0.0028608362, -0.0012800792, 0.029849758, 0.027162999, -0.04132827, -0.010507903, -0.013250929, 0.029258953, -0.04644859, -0.017456904, 0.010979141, 0.020213997, -0.07089669, -0.0036186855, 0.016415961, -0.0021258458, 0.03499821, 0.0042903754, 0.027430268, -0.013996471, -0.027416201, 0.0101703, -0.027767872, -0.004128607, -0.038965046, -0.004905798, 0.035335813, 0.015965823, 0.019665392, 0.0022454136, -0.050640497, 0.0077930097, -0.018244645, -0.009037922, -0.0020572701, 0.009656861, 0.006425013, -0.00712132, 0.031903513, -0.005851791, 0.031312708, -0.048474208, -0.012639023, 0.013187629, -0.012209986, -0.021409675, 0.023252428, -0.016458161, 0.009910064, 0.0018744017, 0.002894245, -0.008918354, -0.014376274, -0.0035254927, 0.017625706, 0.076523416, -0.059699517, -0.04310069, -0.020270266, 0.0004655233, 0.010943973, -0.012821892, -0.03305699, -0.04675806, -0.014214505, -0.034097932, 0.034660608, 0.018638516, 0.016008023, 0.019257456, 0.016795764, -0.025601584, -0.0035694516, 0.03837424, 0.0050499826, 0.033169523, 0.044535503, -0.03288819, -0.007659375, -0.0042622415, -0.00649183, 0.024785709, 0.031228306, 0.023519697, 0.026206456, -0.022000482, -0.06268168, -0.008658118, -0.02253502, -0.039780922, -0.0007938953, 0.017724173, 0.012027117, -0.019777928, -0.01742877, -0.02191608, 0.014826411, 0.0065234806, -0.004870631, 0.008299415, 0.000498932, 0.0052856016, 0.010437569, 0.010929907, -0.011569947, -0.016444094, 0.002628734, -0.019960795, -0.022014549, 0.034126066, -0.044366702, -0.008468216, 0.021058006, -0.0095161935, -0.00034309816, 0.003019088, 0.050387293, -0.017963309, 0.003064805, 0.0012554623, -0.04217228, -0.02007333, 0.023168027, -0.016190892, 0.029906025, -0.037839703, -0.016570695, 0.048839945, -0.0057603563, 0.007434306, 0.0064390795, -0.01910272, -0.029737225, 0.033394594, -0.006509414, 0.00042200414, -0.040287327, -0.032634985, -0.00646018, -0.017161502, -0.0044486267, 0.015459418, -0.0027711606, -0.029090151, 0.008510416, -0.0009266507, 0.014854546, 0.03904945, 0.055873346, 0.023055492, -0.022928892, -0.01593769, -0.00849635, 0.007975878, -0.007047469, 0.00221728, -0.012020084, -0.01918712, -0.008137646, -0.03195978, -0.016852032, -0.03873998, 0.01831498, 0.017161502, 0.0075609074, -0.0036819861, -0.012526489, -0.021508144, 0.02495451, -0.013679967, 0.018891718, 0.011380045, 0.042425483, -0.023407163, -0.03820544, -0.0035290096, -0.017063033, -0.02439184, -0.013757335, 0.0013178837, -0.01852598, 0.016247159, 0.01742877, 0.00038244124, -0.001464706, -0.020143664, 0.009706095, 0.021564411, -0.005268018, -0.01052197, -0.015965823, 0.009037922, -0.008088413, -0.032381784, 0.02170508, 0.007469473, 0.015768887, -0.03195978, -0.03671436, -0.0028625948, 0.027655337, 0.020523468, 0.010859573, -0.0022981642, -0.0043396093, -0.025671918, 0.013250929, 0.014073838, -0.006115543, -0.0057709063, -0.031087637, -0.0069701015, 0.01602209, 0.021184606, 0.0030964552, 0.027669404, 0.01093694, -0.013588533, 0.00018616536, 0.009607627, -0.006315995, -0.03226925, 0.00845415, 0.028625946, -0.018835451, -0.003836721, 0.0059854253, -0.013771402, -0.0014814103, -0.01593769, 0.00040266226, 0.011112776, 0.018582249, 0.011316744, 0.0063230284, 0.0039949724, -0.015993956, 0.01095804, -0.014432541, -0.015909556, 0.013581499, -0.013876903, -0.0035342847, -0.013215763, 0.009537294, 0.0030876636, -0.034041665, 0.011935684, -0.0046139117, 0.007989945, 0.041609608, -0.017175568, 0.020002997, 0.010690771, -0.025545316, 0.02923082, -0.02584072, 0.008222047, -0.009952264, -0.024068302, 0.0086159175, 0.020636002, -0.0078070764, 0.01852598, 0.009769396, 0.012751558, 0.008011045, -0.0037945204, -0.035673417, 0.01434814, 0.00646018, 0.018132111, -0.013229829, -0.009713128, -0.0155860195, -0.026881663, 0.00959356, 0.022197418, 0.010817372, -0.02187388, 0.01672543, -0.013743267, -0.02433557, 0.00041826765, 0.004599845, 0.013553366, -0.035504613, -0.025629718, 0.002393115, 0.010550103, 0.017020833, -0.04543578, -0.0019130853, 0.015445352, 0.015206216, 0.02848528, 0.028808815, -0.020453133, 0.018371247, -0.01335643, 0.03845864, 0.010289867, -0.034323003, -0.0021838713, 0.015909556, 0.027289601, 0.02685353, -0.020495335, -0.0058377236, -0.02852748, -0.034407403, 0.015656354, -0.01335643, -0.024152704, 0.0020414449, -0.06887107, -0.0041110236, -0.009938197, 0.0064074295, -0.0014436058, 0.02954029, -0.026951997, -0.013103228, -0.039190117, 0.022309951, 0.0028520445, 0.008608884, -0.014214505, -0.018568182, 0.024532506, 0.044029098, -0.0020484785, 0.016542561, -0.008362715, -0.00886912, 0.0076664085, -0.01058527, 0.0018585766, -0.0024687243, -0.013996471, 0.015951756, 0.014924879, -0.00014264618, 0.023955768, -0.014643543, 0.0004725567, 0.024152704, 0.014397374, 0.014130105, 0.0028327028, 0.0029135868, -0.0031140388, -0.01333533, 6.874052e-05, -0.008573717, -0.04529511, -0.029765358, -0.016612897, -0.014882679, 0.0041356403, 0.040737465, -0.009839729, 0.029652823, -0.0057111224, -0.019398123, -0.014924879, -0.002212005, -0.01751317, 0.013778435, 0.0105571365, 0.017836709, 0.024307437, -0.010725939, -0.025587518, -0.01050087, -0.0022594805, 0.016289359, -0.008447116, -0.009206723, 0.0026146672, -0.0101069985, -0.01505148, 0.03134084, -0.0052891183, -0.0013873386, -0.0037945204, -0.0025003746, 0.031678442, -0.017921109, -0.011738748, -0.02117054, 0.0010207224, -0.011281577, 0.010325034, -0.006903284, -0.020002997, -0.018413447, -0.008791753, 0.010824406, 0.01870885, 0.037389565, 0.010050732, 0.017189635, -0.04718006, 0.0073358384, -0.00614016, -0.0060452092, -0.0033725163, -0.011605114, 0.007821144, -0.0134197315, 0.00480733, -0.020706335, 0.03446367, -0.058011502, -0.029793492, -0.006080376, -0.0017170293, 0.019341856, 0.0015570194, 0.024138637, -0.0026006005, -0.04476057, -0.010022598, 0.013862835, -0.014981147, 0.02284449, 0.029455889, -0.02094547, -0.008735485, -0.005387586, 0.008172813, -0.024884177, 0.03260685, -0.030299896, 0.029596556, -0.022324018, 0.011140909, -0.029062018, 0.03015923, 0.01091584, -0.003717153, 0.02007333, 0.005872891], "content_hash": "2658a78bbeb4acf810866e700735391e"}, {"content": "# Interpreting Attention Maps\n\n## Quick Reference\n\nThe BertViz attention visualization in Stage 3 shows how tokens attend to each other. Here's how to read the patterns.\n\n## Reading the Visualization\n\nThe BertViz display shows:\n- **Left column**: Each token in your input (the \"query\" -- the token doing the looking)\n- **Right column**: The same tokens (the \"keys\" -- what's being looked at)\n- **Lines between them**: Attention connections. Each line shows how much one token attends to another.\n\n### Line Properties\n- **Thicker, more opaque lines** = stronger attention (the model focuses more on this connection)\n- **Thin, faint lines** = weak attention (some attention, but not much)\n- **No line** = very little or no attention between those tokens\n\n### Interacting with BertViz\n- **Single-click** a head square at the top to select/deselect it\n- **Double-click** a head square to view only that head (deselects all others)\n- **Hover** over a token or line to see exact attention weights\n\n## Common Attention Patterns\n\n### Diagonal Pattern (Previous-Token)\n\nYou see each token strongly attending to the token directly before it, creating a diagonal line of strong connections.\n\n**What it means**: This head tracks local word order. It's useful for bigram patterns -- sequences of two words that commonly appear together.\n\n**Looks like**: A staircase pattern of thick lines, each shifted one position to the left.\n\n### Vertical Stripe (First-Token / Positional)\n\nYou see many or all tokens attending to the same position (usually the first token), creating a vertical column of lines.\n\n**What it means**: The first token often serves as a \"default sink\" for excess attention. This is an artifact of the softmax function -- attention weights must sum to 1.0, so when a head has nothing specific to attend to, it sends attention to the first token.\n\n**Looks like**: Many thick lines all pointing to the same token on the right side.\n\n### Uniform / Diffuse (Bag-of-Words)", "source_file": "interpreting_attention_maps.md", "chunk_index": 0, "embedding": [-0.016242096, 0.03925803, 0.026767982, 0.0037803797, -0.042632356, -0.032005746, 0.018055169, 0.02918541, -0.032962643, 0.024891958, -0.016015463, 0.006729768, 0.016884226, -0.0083791595, -0.014643068, 0.019767513, 0.014303118, 0.03371809, 0.004501201, 0.02199608, -0.019163156, 0.00740967, 0.028631417, 0.05247834, 0.020006739, 0.005398294, -0.0023025377, 0.021744264, 0.035002347, 0.0016777205, -0.02038446, -0.0116464645, -0.014932657, -0.033290002, -0.0033711793, 0.013963167, -0.032232378, 0.0596299, -0.032333106, 0.003745755, -0.019780105, 0.029739406, -0.022927796, 0.028203331, 0.0061411494, 0.0109162005, 0.04580523, 0.01776558, -0.045931138, 0.069954336, -0.025332635, 0.046031866, 0.0024158545, -0.018483255, 0.012729271, 0.00798255, -0.03286192, 0.013887622, 0.053233787, -0.012565591, 0.057262834, 0.002204959, 0.035556342, 0.027120525, -0.023154432, -0.006175774, -0.06940034, -0.0019012067, 0.04640959, -0.018168485, 0.058924817, 0.02181981, 0.02151763, 0.0024473316, 0.04124737, -0.016909407, -0.016896818, 0.007623713, -0.015776237, -0.016720546, 0.043866254, -0.018118123, 0.0014314135, -0.027422704, -0.022059035, -0.06219842, -0.020535551, -0.05917663, 3.5682053e-05, -0.02068664, 0.0022175498, 0.003437281, -0.017400447, 0.011369468, 0.016884226, -0.027473066, 0.009965597, -0.024703097, -0.03638733, -0.014152029, 0.059428446, -0.0043532597, -0.0024190021, -0.035380073, 0.055903032, -0.031023664, 0.028102605, 0.0020223928, 0.0059271064, -0.012823702, -0.069299616, -0.023104068, 0.017790763, 0.03585852, 0.045125328, 0.01776558, -0.027724883, 0.0018665821, 0.048978105, -0.03195538, 0.007799984, -0.03031858, 0.033441093, 0.03860331, -0.016959772, -0.022940388, 0.015247426, -0.02068664, -0.005351078, -0.042733084, -0.00055399403, 0.0022222714, 0.013157357, -0.0121438, 0.0035002348, -0.03470017, -0.003991275, 0.014693432, -0.052981973, 0.00021483006, -0.0019562913, 0.00305641, 0.003981832, 0.0012874065, 0.0036198471, 0.020623686, -0.050212003, 0.040819287, -0.033214457, 0.0011481211, 0.0007684306, 0.022071624, -0.022537483, -0.05207544, -0.022235304, -0.020245962, 0.008712815, -0.0018303836, -0.016984953, -0.010041141, 0.04353889, -0.014189801, -0.009764144, -0.017501174, -0.057212472, 0.040466744, -0.035757795, 0.01971715, -0.01904984, -0.061644424, -0.0074852146, -0.031905018, 0.035657067, -0.028681781, 0.028656598, -0.014063893, -0.064615846, -0.009638237, -0.001499089, -0.037873045, 0.010821769, -0.019200929, 0.00010008681, 0.0018209405, -0.022726346, 0.0020066544, -0.03273601, 0.032559738, -0.028228512, 0.009720077, -0.025508905, 0.007611122, 0.047719028, 0.036865782, -0.01989342, -0.073731564, 0.029034322, 0.010928791, 0.013598035, 0.020623686, 0.007976254, 0.011929757, 0.03323964, -0.028127786, 0.00019466515, 0.008165116, -0.0016368005, -0.015524423, -0.017992215, -0.028908415, -0.001169368, -0.04301008, 0.053082697, 0.011249855, -0.05570158, 0.004164398, -0.032056108, 0.03663915, 0.0060561616, 0.047769394, 0.024350554, -0.004740426, 0.04384107, 0.022184942, -0.03963575, 0.03842704, 0.091056466, -0.031527296, 0.018105531, -0.021580584, 0.033315185, 0.029512772, 0.04429434, 0.017841125, -0.031250298, -0.051042993, 0.02574813, -0.019150566, 0.008347683, -0.012836293, -0.0066982913, 0.0071704453, 0.02476605, 0.015020792, 0.011847917, 0.023557335, -0.028782507, -0.020094873, 0.033164095, 0.037873045, 0.03583334, 0.008630975, -0.067083634, 0.02639026, -0.028429965, 0.021832401, -0.035606705, 0.051521443, -0.0030721487, 0.037495323, -0.005766574, 0.024010604, -0.012439684, -0.034448355, -0.015725875, -0.03976166, -0.012748158, 0.0075544636, -0.07947296, -0.025609631, -0.021089545, 0.019805286, 0.038301133, 0.021630948, -0.038855124, -0.009411602, 0.009978187, 0.0012850458, 0.003251567, -0.024501644, -0.032358285, 0.014315709, 0.057313196, 0.042028, -0.025370408, -0.0029430932, -0.005656405, -0.028732143, -0.0008837149, 0.005045752, 0.032156833, -0.03228274, 0.042103544, 0.016355414, -0.02916023, 0.013333629, 0.044697244, -0.009115719, -0.004236795, -0.031905018, 0.046208136, -0.06270205, 0.01301886, 0.038527764, 0.026339896, -0.0033082257, 0.010689566, -0.017337494, -0.007113787, 0.00508982, -0.038074497, -0.016493913, -0.021102136, -0.033441093, -0.0859698, -0.00863727, 0.03422172, 0.07342939, 0.0035537456, -0.016279869, -0.0044162134, -0.020094873, 0.02200867, 0.049179558, 0.017085679, -0.02149245, -0.026692439, 0.013333629, 0.0069689928, -0.006773836, 0.046334043, 0.0060183895, -0.010481819, 0.0052409093, -0.05182362, -0.022222714, -0.0039220257, 0.048801836, -0.016594639, 0.08184002, -0.047341306, -0.018911341, 0.0038181518, 0.009468261, -0.048197478, 0.027347159, 0.0029430932, -0.015927328, -0.018118123, -0.016821273, -0.03046967, 0.020585913, 0.014529752, -0.058874454, -0.005731949, 0.03064594, -0.030419307, -0.020913273, -0.02184499, 0.0050709336, 0.011092471, -0.017123451, 0.026264353, 0.017022725, 0.0094053075, -0.042430904, -0.008819836, -0.0037205734, 0.010884723, -0.020183008, 0.017652264, 0.018344756, -0.02363288, -0.024174284, -0.04872629, -0.014680841, -0.0039440594, -0.0022946685, -0.033466276, 0.0387544, -0.03898103, 0.001702902, -0.030520033, -0.021895355, 0.01531038, 0.0059869126, -0.016544275, -0.0020711822, -0.05983135, -0.026188808, -0.017375266, 8.749603e-05, -0.013799487, 0.056003757, 0.024325373, 0.021933127, -0.0029399456, 0.015184472, 0.016065827, 0.05278052, 0.03225756, 0.00067911483, -0.03487644, 0.0003051295, 0.003487644, 0.009682304, -0.019339427, 0.015990281, 0.0035380071, 0.020510368, 0.04076892, -0.015990281, 0.0035065303, -0.010047437, -0.026541349, 0.023695834, 0.012848884, -0.008599498, 0.020573324, 0.044797968, -0.01565033, -0.023015933, -0.035782978, 0.00019663246, -0.06436403, 0.0023104069, 0.052881245, -0.021051772, 0.017501174, -0.014970429, 0.006635337, 0.05832046, 0.02739752, 0.011306514, -0.00053117325, -0.034246903, 0.024841595, 0.03046967, -0.016859045, -0.036034793, -0.0046837674, -0.03764641, -0.046006683, -0.004482315, -0.0150711555, 0.0077810977, -0.016065827, -0.045553416, 0.050161637, -0.03187984, 0.006503134, 0.044546153, -0.0075985314, 0.04900329, -0.026944254, 8.9660076e-05, 0.051042993, 0.0064181467, 0.012848884, -0.042909354, 0.024816412, 0.008977221, -0.03046967, 0.03162802, -0.012320071, -0.00071964134, 0.0036544718, -0.021404313, 0.07685408, 0.015914736, 0.0017359528, -0.012653727, 0.029714223, -0.015234835, 0.009965597, -0.036286607, 0.016632412, -0.036236245, 0.0035443027, -0.0361607, 0.049229924, 0.022839662, -0.02427501, -0.0016950328, 0.020170419, 0.05197471, -0.028278876, -0.035228983, 0.029563135, 0.0030910347, -0.02704498, -0.040164564, -0.0028659746, -0.012861474, 0.008032913, -0.011627579, -0.02067405, -0.04416843, 0.013044041, -0.04839893, 0.031552475, -0.025861448, 0.01661982, 0.0167835, 0.027649337, 0.0023434577, 0.033264823, 0.018029986, -0.020623686, 0.04578005, -0.01824403, -0.023192203, -0.008121048, 0.018281803, -0.033592183, 0.03258492, 0.0033774748, 0.03240865, 0.00023922468, -0.0012299612, -0.007019356, -0.0021010851, 0.01531038, -0.007755916, 0.007674076, -0.011457603, -0.038905486, -0.011331695, -0.04802121, -0.029739406, 0.008819836, -0.031980563, 0.0003065066, -0.00042847975, -0.00092306104, -0.0008577464, -0.013723942, -0.027422704, -0.0145045705, 0.037016872, -0.007264876, 0.007674076, 0.02996604, 0.043589257, -0.005980617, 0.029009141, 0.008857609, -0.006210399, 0.028127786, 0.055500127, -0.031426568, -0.040416382, 0.0044067707, -0.0115898065, -0.020422233, -0.023658061, -0.030520033, -0.021404313, 0.0026897038, -0.04190209, 0.028077424, -0.035707433, -0.026919072, -0.020913273, -0.0125089325, 0.047844935, 0.03487644, -0.017576719, 0.022537483, -0.031703565, 0.0031146423, -0.01155833, 0.034926806, -0.03145175, -0.0026283239, -0.016745728, -0.036034793, 0.01661982, -0.018105531, 0.015335562, -0.045125328, 0.006937516, -0.02574813, -0.014605297, 0.022701163, -0.009801917, 0.031502113, -0.025534088, -0.017198997, -0.011772373, 0.060838614, -0.0057508354, -0.010318139, 0.038049314, -0.017627083, -0.011098767, -0.015411106, -0.014454207, -0.030394126, 0.032635283, 0.0043123397, -0.000718461, -0.0121563915, -0.002492973, -0.016670182, 0.057615377, -0.051445898, 0.0019767513, 0.0022380098, 0.036286607, -0.037873045, -0.029537953, -0.034448355, -0.021567995, -0.028203331, 0.028656598, -0.057615377, 0.0040982966, -0.016418368, 0.010311843, 0.01181644, 0.0050205705, 0.004875777, 0.0028455146, 0.0054643955, 0.04321153, 0.024526825, -0.030117128, -0.005011128, -0.036513243, -0.055852666, 0.0014219704, 0.020183008, -0.004614518, -0.0026456362, -0.0075481683, 0.0072963526, 0.014454207, 0.026163626, -0.0038559241, -0.0201956, 0.012647431, -0.019968966, -0.01645614, 0.01971715, -0.027145706, -0.001251208, -0.028278876, -0.02330552, -0.02052296, -0.023431428, -0.040189747, 0.043287076, -0.0045924843, -0.00080817024, -0.033038188, -0.037570868, -0.019767513, -0.006389817, 0.030368945, -0.008007731, 0.030444488, -0.0027321978, -0.043916617, -0.023758788, 0.005225171, 0.012231936, 0.014429025, 0.012553001, 0.0135350805, -0.052176163, 0.018911341, 0.029714223, 0.020787366, 0.0095941685, 0.010544772, 0.016657593, -0.03956021, -0.04774421, 0.018949114, 0.0060152416, 0.016833864, -0.039862387, -0.034951985, 0.020535551, 0.024564598, 0.030167492, -0.012483751, 0.019855648, -0.027825609, 0.006906039, -0.015260017, 0.044747606, -0.006336306, -0.017010134, 0.013094404, 0.025685176, 0.048449293, -0.030368945, -0.070155784, 0.010702156, 0.004151807, -0.013270674, -0.012559296, -0.014152029, -0.0042588287, 0.03963575, 0.03517862, 0.008876495, -0.042430904, -0.04545269, 0.0036859487, -0.020233372, 0.020900683, -0.024463871, 0.042103544, -0.037042055, 0.011614988, 0.0008726979, -0.0041895797, -0.027347159, 0.030998483, -0.031426568, -0.013635807, 0.028152969, 0.011306514, -0.039937932, -0.035304528, 0.03784786, 0.03273601, 0.0064905435, 0.054190688, -0.0029289285, -0.06582456, -0.019012067, 0.023972832, -0.026818346, -0.0010828065, -0.023746198, -0.018596571, 0.0011127095, -0.013774306, 0.0065723835, -0.0019625868, 0.0037174257, 0.018659525, -0.024149103, 0.0024898252, -0.05917663, -0.009059061, 0.0016320789, 0.009499738, 0.007195627, -0.010072619, -0.0016305051, -0.001743822, 0.011829031, -0.017236767, 0.035405252, 0.040517107, 0.016934589, -0.0075041004, -0.030343762, 0.0011394649, -0.0062355804, -0.002113676, 0.020258553, 0.006566088, 0.037721954, 0.02022078, 0.014441617, -0.005640666, 0.035077892, 0.020371871, 0.07428556, -0.0019326836, 0.05343524, -0.004186432, -0.010771406, -0.00813364, 0.0048285616, -0.019805286, -0.019855648, -0.04235536, 0.026289534, 0.012093437, -0.03817522, -0.05094227, -0.0006248171, -0.013069222, -0.03094812, -0.03631179, 0.015662922, -0.044546153, 0.0012142226, 0.0129936775, -0.004328078, -0.10576249, -0.025332635, 0.010563659, -0.046006683, -0.033944722, -0.002894304, 0.03729387, -0.024186874, 0.0032578625, -0.014920066, 0.00044854629, 0.007478919, -0.022550074, -0.013749124, 0.023972832, -0.033642545, -0.022184942, -0.00031162164, -0.0227893, 0.011369468, -0.044218794, 0.0071326727, 0.0012559296, 0.028253695, -0.01548665, 0.036437698, -0.026490986, 0.004712097, 0.03306337, 0.025445951, -0.027775245, 0.025043048, -0.00033109798, 0.04822266, 0.017085679, 0.017123451, 7.84956e-05, 0.060385346, 0.025508905, -0.008561726, 0.01335881, -0.036085155, 0.008889086, -0.032182015, 0.049431372, 0.009235332, -0.032156833, -0.021769445, -0.012364139, 0.016720546, -0.035002347, 0.0028895824, 0.0057130633, 0.021731675, 0.015134109, 0.0345239, -0.015537013, -0.024816412, 0.0125341145, 0.018168485, -0.07121341, 0.005388851, 0.05998244, 0.003877958, 0.016229507, -0.012320071, 0.009317172, 0.02671762, -0.007856642, -0.004164398, -0.0019783252, 0.004359555, 0.0068367897, -0.02509341, 0.009789326, -0.007932187, 0.03454908, 0.002760527, -0.0062544663, -0.010104096, 0.050715633, -0.014580115, 0.005517906, 0.023670653, 0.018621754, 0.014378662, 0.018395118, -0.02231085, -5.523218e-05, 0.049582463, -0.007976254, 0.0030501147, 0.00075977447, -0.0047781984, 0.02134136, -0.022738935, 0.025521496, 0.032307923, 0.09014993, -0.008114753, 0.03585852, 0.012842588, 0.0007294779, -0.033843998, -0.042607177, 0.011419831, 0.02003192, -0.019931193, 0.002740067, -0.03031858, 0.00258583, 0.013245493, 0.047819756, 0.012074552, -0.009839689, 0.028807689, 0.06239987, 0.023167022, 0.014731204, -0.025722949, -0.0056752907, 0.0070004696, -0.008221774, -0.020132646, -0.006292239, -0.011256151, 0.009644532, -0.010053732, 0.0003472299, -0.013711352, 0.0023733606, 0.018621754, -0.009745258, -0.0066605187, -0.017450811, 0.012842588, -0.014189801, 0.0345239, -0.011369468, 0.023519564, -0.03177911, 0.00781887, 0.035581525, 0.025269682, -0.01483193, 0.014718614, -0.009367535, -0.035279345, 0.016028054, 0.026566532, 0.05003573, 0.027750064, 0.03225756, 0.0015518127, 0.0007641025, 0.031804293, 0.0014227573, -0.04872629, -0.008246956, 0.04849966, -0.021139907, 0.014403844, 0.056154847, -0.0047435737, 0.036362153, -0.003950355, -0.026591713, 0.0035443027, -0.0047561643, 0.034901623, 0.0061474447, -0.018508436, 0.033692908, 0.04562896, 0.014920066, 0.01970456, -0.0033837701, -0.013598035, 0.03177911, 0.02231085, 0.0051118536, 0.027598973, -0.02981495, 0.005766574, 0.02378397, -0.007995141, -0.009336058, 0.0064401804, -0.008612089, -0.04787012, 0.028555872, 0.02165613, -0.034397993, 0.016632412, 0.015234835, -0.0074222605, 0.01694718, 0.0025071376, 0.0071767406, 0.034095813, 0.0023938208, 0.005993208, 0.02199608, -0.02280189, -0.011199492, 0.012376729, 0.004759312, -0.018042577, 0.00035116452, 0.0033082257, -0.02166872, 0.027447885, -0.039862387, -0.00026971797, -0.0050331615, 0.007850347, -0.0050237183, -0.017249359, 0.0057571307, 0.022348622, -0.042405725, -0.03399509, -0.01139465, 0.009638237, 0.0046900627, 0.032811552, -0.0075041004, -0.015335562, -0.018583981, -0.008542839, -0.018193666, 0.019125383, -0.0161036, 0.011300218, 0.0032295333, 0.0055525308, 0.03809968, 0.0038496288, 0.029361682, 0.0114827845, 0.04142364, 0.0066038603, 0.0031020516, 0.010230003, -0.029537953, -0.041751, 0.0069941743, -0.011961234, 0.0046554385, 0.022449348, -0.025017865, 0.01565033, 0.019163156, 0.00014813831, -0.02086291, -0.0054738387, 0.030822212, 0.023028523, 0.008372864, 0.037747137, 0.039006215, 0.036588784, 0.0018697297, -0.039837204, -0.01742563, 0.009953006, -0.02948759, -0.019238701, 0.00749151, 0.025269682, -0.017639672, 0.03341591, 0.010828065, -0.007460033, 0.013459536, 0.024665324, -0.00398498, -0.0046396996, 0.046661403, -0.02623917, 0.010576249, -0.0485752, 0.030268218, -0.034120996, 0.026944254, 0.018004805, -0.026566532, -0.031577658, 0.017853716, -0.028329238, 0.0098900525, -0.0055556786, 0.015411106, -0.021102136, -0.016368005, -0.004350112, -0.032685645, -0.0013354089, -0.0022096806, -0.032383468, 0.010362206, 0.033315185, 0.038477402, 0.0034341333, 0.00266767, 0.016091008, 0.03485126, -0.025445951, -0.018029986, 0.018181076, 0.023431428, 0.034322448, -0.019490516, 0.030520033, -0.008196593, -0.011545739, 0.025005275, -0.03162802, 0.016796092, -0.02445128, -0.00521258, 0.058874454, -0.014630478, 0.01075252, 0.0007275106, -0.006553497, 0.0042210566, -0.0031791702, 0.013106994, 0.0072145127, 0.011526853, 0.013799487, 0.00749151, 0.020170419, -0.04170064, -0.0019657344, -0.006021537, -0.011665351, 0.027095344, -0.031351026, 0.030066766, 0.010041141, -0.007932187, -0.006122263, 0.04447061, -0.0076866667, 0.025156364, 0.00034290183, -0.026667258, 0.019251293, 0.06204733, -0.040844467, -0.017967032, 0.018445482, 0.003478201, 0.012181573, -0.0074852146, 0.01180385, -0.03930839, -0.0260629, -0.0074222605, -0.0052157277, 0.0011394649, -0.03988757, 0.009984483, -0.030041585, -0.020925865, 0.023519564, 0.03845222, 0.03112439, 0.018659525, 0.029462408, -0.0401142, 0.0046522906, 0.009077948, 0.028908415, 0.0045484165, 0.0029682748, 0.010148163, 0.026667258, -0.017967032, -0.011344287, -0.0067423587, -0.021064363, -0.022386394, 0.005684734, -0.018949114, 0.019528288, -0.010122981, -0.002563796, 0.011961234, 0.014000939, -0.0038212996, 0.012445979, -0.0091346055, -0.003723721, 0.017664853, 0.014743795, -0.03399509, -0.0150711555, -0.0056658476, 0.043412983, 0.0064464756, -0.026365079, 0.013396583, -0.03306337, 0.0015950935, 0.006629042, -0.006975288, -0.015537013, 0.007006765, 0.029512772, -0.006081343, -0.016116189, 0.022109397, -0.033818815, -0.0045074965, -0.015348152, -0.026591713, 0.013006268, -0.028203331, -0.007541873, 0.0025386147, 0.0017406744, 0.03747014, -0.0026424883, 0.014403844, -0.0014282658, 0.010053732, -0.03747014, -0.009556397, -0.0014093796, -0.054996494, 0.028807689, -0.020460006, 0.0035348595, 0.02817815, -0.041977637, -0.015184472, 0.033642545, -0.024854185, 0.01775299, -0.004318635, 0.022613028, 0.027422704, -0.019465335, 0.017337494, -0.025181545, 0.003764641, -0.006330011, 0.0067360634, -0.008083276, -0.015562195, 0.005857857, 0.018697297, -0.010991745, -0.032937463, 0.016179143, 0.026994618, -0.009883757, -0.0009151918, -0.016884226, 0.023393655, -0.015197063, -0.008983516, 0.030091947, 0.0030217855, 0.0319302, -0.034322448, -0.011602397, 0.02396024, -0.009814507, -0.0035820748, -0.028404783, 0.0030926086, 0.029210594, -0.009084242, 0.018193666, -0.039283212, -0.008750587, -0.026944254, 0.0016871636, 0.036236245, 0.007541873, -0.009921529, 0.021379132, -6.374079e-05, 0.0084609995, -0.047668666, 0.026944254, 0.02441351, 0.0047750506, -0.013547672, -0.01776558, 0.03243383, -0.001918519, 0.016909407, 0.019666787, -0.0002201418, 0.0025401884, 0.00093958643, -0.04414325, 0.0033711793, 0.017450811, 0.01954088, -0.03794859, -0.013761715, -0.004110887, -0.016607229, -0.0046963585, 0.013711352, 0.009474557, -0.022348622, -0.0052849767, 0.034826078, 0.015335562, -0.016468732, -0.0075229867, 0.02885805, 0.033189278, -0.00035706646, -0.010884723, -0.009896347, 0.016242096, 0.0022364361, 0.013081813, 0.01270409, 0.007183036, 0.013081813, 0.007321534, -0.0021687606, 0.00027601334, -0.01499561, -0.050564542, 0.007233399, -0.0079195965, -0.028606236, -0.031174753, -0.0048191184, -0.00749151, 0.018067759, -0.0050016847, 0.010299252, 0.0046617338, 0.015851783, 0.068695255, -0.010286662, 0.027070161, 0.016745728, 0.025534088, -0.007579645, -0.029537953, -0.013396583, -0.00041313472, -0.018357348, 0.021253224, -0.036840603, 0.015411106, 0.0012480604, 0.0021797775, -0.0032893394, -0.011696828, -0.016582048, -0.0047152443, -0.00058940554, 0.025647404, 0.026088081, 0.018332165, 0.015020792, -0.008335091, 0.02200867, -0.011451308, 0.021316178, -0.00035411547, -0.009713781, 0.047316123, -0.018583981, -0.021014, 0.007415965, -0.040945195, 0.00079400564, -0.045402326, -0.0044791675, 0.01824403, 0.025068229, 0.006204103, -0.0061033773, 0.02135395, 0.002574813, 0.02068664, -0.016191734, 0.028958777, -0.007346716, 0.0051748077, -0.010122981, 0.014114256, -0.021794628, 0.018986885, -0.011684237, 0.003078444, -0.013774306, 0.00863727, -0.028782507, -0.037117597, -0.011363173, -0.028732143, 0.007378193, 0.0018870421, 0.012357843, -0.0020381315, -0.03925803, 0.0022757822, 0.007875528, 0.018974295, -0.0017579866, 0.024891958, 0.014466798, -0.00480338, -0.02966386, 0.0007373471, -0.029462408, -0.0046806196, -0.017639672, 0.03162802, -0.007963664, 0.04545269, 0.012326366, 0.008819836, -0.012074552, 0.021895355, 0.029563135, -0.026591713, 0.0008490902, -0.013044041, 0.0017485436, 0.0052157277, 0.0016667036, 0.0014117404, 0.01042516, -0.012496342, 0.026365079, 0.014366072, -0.004749869, -0.031552475, 0.0015407958, 0.0013204573, -0.014353481, -0.022096807, -0.028807689, 0.018432891, -0.028631417, 0.0037300165, 0.023985423, 0.0028171854, 0.0038936965, 0.011086175, 0.010702156, 0.035455618, -0.016368005, 0.01626728, 0.017992215, -0.001456595, -0.014945247, -0.0022332883, -0.0046932106, -0.017211586, -0.002656653, -0.028455148, 0.017866306, 0.018521028, -0.0016588343, 0.019666787, 0.027573792, -0.01581401, 0.034599442, -0.03817522, 0.0018886159, 0.017060498, 0.034020267, 0.011092471, 0.013497309, 0.023431428, 0.05625557, 0.013472127, 0.0020066544, 0.0061946604, -0.0026802607, 0.0058956295, 0.01124356, -0.031527296, -0.010595135, -0.022575255, -0.039988294, -0.028883234, 0.0042966013, 0.01661982, -0.012364139, -0.018055169, -0.016418368, 0.007787393, 0.01548665, 0.016393187, -0.033038188, -0.03940912, 0.0017878897, 0.02344402, 0.028354421, 0.026616894, -0.023494381, -0.027095344, -0.026667258, 0.01319513, -0.029512772, -0.021970898, -0.045049787, 0.01058884, -0.02885805, -0.03177911, -0.0095941685, 0.012175277, 0.00590822, 0.010154459, 0.018470664, 0.012263413, -0.014957838, -0.011048404, -0.0063961125, -0.025382997, 0.01972974, -0.005838971, 0.031048845, -0.02035928, 0.003017064, -0.009619351, 0.004192727, -0.016758319, -0.024375737, 0.03930839, -0.027271613], "content_hash": "240077b750347b62acd28ee208fffecd"}, {"content": "an artifact of the softmax function -- attention weights must sum to 1.0, so when a head has nothing specific to attend to, it sends attention to the first token. **Looks like**: Many thick lines all pointing to the same token on the right side. ### Uniform / Diffuse (Bag-of-Words)\n\nYou see many thin lines spreading from each token to many other tokens, with no strong focus on any particular one.\n\n**What it means**: This head is gathering a broad summary of the entire input, rather than focusing on specific relationships. It helps the model maintain an overall sense of context.\n\n**Looks like**: A dense web of thin, similarly-weighted lines.\n\n### Structured Connections (Syntactic)\n\nYou see specific, purposeful-looking connections that skip across tokens -- like a token attending to a word several positions away in a consistent pattern.\n\n**What it means**: This head may be tracking grammatical relationships. For example, a verb attending to its subject, or a pronoun attending to its antecedent.\n\n**Looks like**: A few thick lines making specific connections, often spanning several token positions.\n\n### Mixed / Context-Dependent\n\nSome heads show patterns that change based on the input. They might show one pattern for factual prompts and another for creative prompts.\n\n**What it means**: These heads are flexible and context-sensitive. They don't have a single fixed pattern but adapt to the input.\n\n## Tips for Reading Attention Maps", "source_file": "interpreting_attention_maps.md", "chunk_index": 1, "embedding": [0.02615375, 0.03889316, 0.04049613, 0.021190163, -0.016999938, -0.046767406, 0.017576445, -0.005765074, -0.008549183, 0.030850178, -0.0025925257, 0.016887449, 0.00089288346, 0.01739365, -0.004418718, -0.0072274343, -0.0124933375, 0.0060814503, 0.008731978, -0.003624263, -0.024269557, 0.008035951, 0.018110769, 0.060969174, 0.015045437, 0.013822117, -0.020529289, 0.03574346, 0.03259376, 0.023510253, -0.01642343, -0.029697163, 0.0077406666, -0.04409579, -0.020037148, 0.03051271, -0.014651725, 0.05666646, -0.0061341794, 0.010159185, 0.0054662745, 0.027306765, -0.024972614, 0.007195797, 0.023397764, -0.025717854, 0.017759241, -0.0015853954, -0.037993245, 0.0695746, -0.047273606, 0.07564902, -0.0035662607, 0.02297593, -0.017182734, 0.015762556, 0.0022409968, 0.031834457, 0.012711286, -0.009231149, 0.061812844, -0.006285337, 0.017098365, 0.039764952, 0.00090606575, -0.012901111, -0.09066634, -0.0010080092, -0.0027612597, 0.015425089, 0.0768864, 0.033353064, 0.0104615, 0.0010660114, -0.0030530286, 0.00030692868, -0.012359756, -0.0005145504, -0.04547378, -0.030850178, 0.040018052, -0.010363072, -0.003733237, -0.06237529, -0.0013779934, -0.03833071, -0.038949404, -0.043055262, -0.025703793, -0.018110769, -0.011354384, 0.013449496, 0.015003254, 0.022413483, 0.015509455, -0.004647212, -0.004021491, -0.0018209198, -0.034590445, 0.0023376672, 0.03866818, -0.0059478693, -0.008816345, -0.03295935, 0.040468007, -0.03526538, 0.005237781, -0.062769, -0.020683961, -0.023018112, -0.046486184, -0.0036910535, 0.0011978347, 0.07283679, 0.048004787, 0.011009885, -0.04924217, 0.022076016, 0.03965246, -0.0061728475, 0.019193478, -0.015551639, 0.047807932, 0.0350404, -0.020655839, -0.009814687, 0.033831142, -0.021302652, -0.0071184603, -0.01520011, 0.020824572, -0.0013850239, 0.0016100024, -0.04572688, -0.027658295, -0.022483788, -0.021611998, 0.031159522, -0.055597812, 0.0108763045, -0.0016609741, -0.014595481, 0.015186049, 0.028473841, 0.023721172, 0.020880817, -0.040186785, 0.04938278, -0.031243889, 0.030456465, -0.0034432258, -0.00209687, -0.04426452, -0.044601988, -0.018082647, -0.06237529, 0.0020283218, 0.018588848, -0.01070757, -0.025366327, 0.03833071, -0.0058670174, 0.005782651, 0.0078601865, -0.0072555565, 0.035771582, -0.030315854, -0.004714003, -0.009920145, -0.051604442, -0.02103549, -0.014321288, -0.0057896813, -0.011157528, 0.030962666, -0.029528428, -0.06215031, -0.024578901, -0.024522657, -0.031300135, 0.011705913, -0.011748096, -0.0039300933, 0.024888247, -0.045501903, -0.037177697, -0.011002854, 0.048932824, 0.0020810512, 1.1465884e-05, -0.030147118, 0.0065454682, 0.061475374, 0.035968438, 0.0073891375, -0.07379295, 0.03965246, 0.02613969, 0.0031760638, -0.003831665, 0.009013201, 0.015636006, 0.03281874, -0.02017776, 0.004046098, 0.036080927, 0.0043589585, -0.023369642, -0.020093393, -0.019896537, 0.012662072, -0.075930245, 0.037880756, 0.04718924, -0.05677895, 7.618291e-05, -0.018167013, 0.030315854, 0.0024905824, 0.05326366, 0.026477158, 0.017913913, 0.06828097, 0.04682365, -0.068730935, 0.046514306, 0.064737566, -0.016873388, 0.0012734134, -0.020262126, 0.034140486, 0.020374615, 0.00972329, -0.035940316, -0.048257887, -0.07238683, 0.018420115, -0.013224518, 0.06040673, -0.011038008, -0.008956957, -0.019868415, 0.023397764, 0.01677496, -0.0060111443, 0.031159522, -0.027742662, -0.012542551, 0.028909737, 0.021949464, 0.09505342, 0.011108314, -0.0763802, 0.018560726, -0.051041998, 0.028783187, -0.03987744, 0.046879895, -0.004415203, 0.020149637, -0.0067880233, 0.028220741, -0.01958719, -0.005286995, -0.015396967, -0.036193416, 0.014117401, 0.020641778, -0.048567235, -0.023791477, -0.004137495, 0.014356441, 0.01483452, 0.0069637876, -0.045530025, -0.018012341, -0.033324942, -0.01874352, -0.025717854, 0.0011486207, -0.037965123, -0.023496192, 0.068055995, 0.04502382, -0.01617033, 0.009174905, -0.05967555, -0.0204168, -0.008050012, 0.0036172324, 0.030597076, -0.07266805, 0.04597998, 0.0055998554, -0.0042359233, -0.005448698, 0.00662632, 0.007635208, -0.006995425, -0.035237256, 0.04561439, -0.04049613, -0.008900712, 0.0119590135, 0.036193416, 0.0057439823, 0.010032635, -0.025774099, 0.014145523, -0.020852694, -0.039118137, 0.011607485, -0.006594682, -0.018321687, -0.067156084, -0.012387879, 0.0347873, 0.036221538, -0.0072274343, 0.01971374, -0.0591131, -0.017281162, 0.015509455, 0.008155471, 0.028052008, 0.00036031712, -0.01872946, 0.027630173, -0.022708768, -0.0048300074, 0.024958553, 0.0038949403, 0.0097443815, 0.03672774, -0.0108763045, -0.015157927, 0.010531806, 0.03734643, -0.00985687, 0.035715338, -0.078123786, -0.049804617, -0.038021367, 0.0117551265, -0.03987744, 0.04257718, 0.0013560228, 0.0061341794, -0.0019791077, -0.00206699, -0.0016196695, -0.003539896, 0.014412685, -0.038021367, 0.0144408075, 0.049045313, -0.03160948, -0.009498311, -0.011023947, 0.0103419805, -0.02019182, -0.028516024, 0.00668608, 0.0060005984, -0.015059498, -0.018560726, -0.016100025, 0.00937176, 0.0058810785, -0.022652524, 0.00252222, 0.035574727, -0.026941177, -0.031946946, -0.04502382, 0.006394311, 0.00011995924, -0.010524776, -0.0352935, 0.015776617, -0.019910598, 0.0062818215, -0.04544566, -0.013927575, 0.029022226, 0.017281162, -0.0028403536, 0.017351467, -0.0381901, -0.054416675, -0.027939517, 0.0021847521, -0.0071817357, 0.037712023, 0.0126972245, 0.028276986, 0.018138891, 0.022441605, 0.041396044, 0.033606164, 0.038105734, 0.012127748, -0.027897334, -0.009716259, 0.012472246, 0.007515688, -0.00094473397, -0.001705794, 0.021246407, 0.034871668, 0.08487314, 0.013273732, 0.02723646, 0.014250982, 0.007438352, 0.016859327, -0.00034999097, 0.021780731, 0.020430861, 0.03782451, 0.006809115, -0.036530886, -0.032621883, 0.015593822, -0.042352203, -0.0024712482, 0.066931106, -0.04412391, 0.03040022, -0.023552436, -0.00011973953, 0.050901387, 0.0152422935, -0.012064472, 0.009224119, -0.043223996, 0.018307626, 0.03858381, 0.0063486123, -0.08009235, 0.012676133, -0.015875045, -0.06979958, -0.0068161455, -0.043111507, 0.021105796, -0.012373817, -0.03968058, 0.017421773, -0.031047033, 0.006355643, 0.023102479, -0.005620947, 0.041621022, -0.031159522, 0.011115344, 0.04451762, 0.0067880233, 0.039314993, -0.05604777, 0.013477619, -0.029697163, -0.023664925, 0.002304272, -0.02553506, 0.021752609, 0.01847636, -0.05447292, 0.053432394, 0.048454747, 0.007733636, -0.0029581157, 0.0238196, 0.0015643036, 0.035434116, -0.032537516, 0.031215766, -0.02833323, 0.027686417, -0.056582093, 0.039596215, -0.0017286434, -0.0016530647, -0.011895739, 0.014384563, 0.07598649, -0.006225577, 0.0034098304, 0.051520076, 0.025563182, -0.020346493, -0.036024682, 0.012071502, -0.017013999, 0.054416675, -0.02114798, -0.03172197, -0.044405133, -0.004341382, -0.045389414, 0.03892128, -0.009941238, -0.0071114297, 0.0035768067, 0.025225714, 0.022750951, 0.008992109, 0.038386956, -0.017942036, 0.010869274, 0.0050374093, 0.00016730578, -0.005195597, 0.047864176, -0.036896475, 0.0060111443, 0.013365129, 0.027827028, -0.0025591305, 0.000561128, -0.02907847, 0.00061341794, -0.009083507, 0.009259271, 0.013322946, -0.011052069, -0.025619427, -0.007234465, -0.045051944, -0.013632291, 0.00936473, -0.04949527, 0.010124032, 0.024494534, -0.030147118, 0.009899054, 0.008956957, -0.057819474, -0.004583937, 0.031047033, 0.003297341, -0.003638324, 0.021977587, 0.011832463, 0.0051850514, 0.017702997, 0.032284416, -0.013772903, 0.016367186, 0.032425027, 0.010081849, -0.030484587, 0.0076211467, -0.014026004, -0.012570674, -0.021738548, -0.019249722, -0.006904028, 0.010306828, -0.024592962, 0.041114822, -0.040664863, -0.013702597, 0.0012584735, -0.036783986, 0.030231487, -0.0104615, -0.007649269, 0.033381186, -0.049326535, -0.02189322, -0.027812967, 0.012465215, -0.025141347, -0.0011837736, -0.018138891, -0.020346493, 0.010117002, 0.0046542427, 0.01567819, -0.02238536, -0.0033746776, -0.037740145, -0.019193478, -0.00754381, -0.012169931, 0.03748704, -0.016957754, -0.0260272, 0.02019182, 0.039708704, -0.00985687, -0.0016108812, 0.04949527, -0.032256294, 0.002225178, -0.0045663607, -0.013217487, -0.014426746, 0.013266701, 0.009329577, 0.0032358237, -0.0009860386, 0.001963289, -0.023411825, 0.044826966, -0.07930492, 0.03200319, 0.039455604, 0.02456484, -0.019854352, -0.010770846, -0.015776617, -0.010032635, -0.021921342, 0.03574346, -0.037262063, -0.0013630533, -0.0055611874, 0.00074831716, 0.0014026003, -0.011565302, 0.037740145, 0.019910598, 0.001726007, 0.049185924, -0.004682365, -0.023566497, -0.0038105734, -0.011902769, -0.06833722, 0.003757844, 0.022807196, 0.009582678, -0.021668242, -0.0014447838, -0.0023359095, -0.007705514, 0.025281958, -0.018167013, 0.0013902968, -0.014553296, -0.015481333, -0.009554556, 0.044798844, -0.002893083, -0.02348213, -0.013829147, -0.033493675, -0.024508595, 0.006225577, -0.046401817, 0.03245315, 0.016240636, 0.0012505641, -0.036080927, -0.02114798, 0.00019806458, 0.004218347, 0.008120318, -0.019263783, 0.009624861, -0.020163698, -0.03830259, -0.018560726, 0.017056182, 0.009842809, 0.0042745913, 0.008289051, 0.004714003, -0.08768537, 0.03878067, 0.017857669, 0.03175009, 0.025942834, 0.00055849156, -0.0005286116, -0.025563182, -0.03613717, 0.0035785644, 0.010363072, 0.039933685, -0.030343976, -0.03284686, 0.019080989, 0.02710991, 0.050648287, 0.009139752, 0.036812108, -0.026730258, 0.00063758553, -0.008542152, 0.029725285, -0.013554955, 0.012971417, -0.019474702, 0.0130136, 0.05067641, -0.02128859, -0.011572332, 0.002863203, 0.011741065, -0.018771643, -0.0014948767, 0.0051463833, -0.014539235, 0.022933746, 0.041564777, 0.006981364, -0.02737707, -0.031665724, -0.0056033707, -0.0010405256, 0.035209134, 0.0052975407, 0.02784109, -0.028403535, -0.021597937, -0.004647212, 0.001355144, -0.029500306, 0.0052026277, -0.014426746, -0.0030829085, 0.022483788, -0.013702597, -0.015171988, -0.030540831, -0.006288852, 0.02224475, -0.017773302, 0.02491637, -0.0029581157, -0.04533317, -0.023004051, 0.035715338, -0.01969968, -0.008767131, -0.033887386, -0.02284938, 0.03669962, -0.007916431, 0.003659416, -0.00681263, 0.013442465, 0.042999018, -0.007213373, 0.0003750374, -0.062769, -0.016690593, 0.001790161, 0.013175303, 0.019952781, 0.003975792, -0.00010990771, -0.0072977403, 0.0016557012, -0.0021882674, 0.030850178, 0.023622742, -0.011895739, 0.0065208613, -0.021372957, 0.014222859, 0.016395308, -0.02868476, -0.0031672756, -0.011354384, 0.018040463, 0.0012918687, -0.0007259072, 0.015228232, 0.018377932, 0.015875045, 0.03720582, -0.018870072, 0.037993245, -0.041986614, -0.03864006, -0.015453211, 0.02992214, -0.023383703, -0.0028350807, -0.0036805077, 0.005265903, -0.0050795926, -0.029134717, -0.03222817, 0.0009948268, -0.015846923, -0.016268758, -0.027138032, -0.003659416, -0.03658713, 0.0033201906, 0.021049552, 0.006489224, -0.09471595, -0.043505218, 0.019235661, -0.040552374, -0.032621883, -0.020641778, 0.032537516, 0.023636803, 0.026603708, -0.021626059, -0.0028016854, 0.019024745, -0.040158663, -0.006724748, 0.021626059, -0.05227938, -0.043702073, 0.014609542, -0.020248065, 0.002690954, -0.044911332, -0.0029247205, 0.005993568, 0.014665786, -0.023580559, 0.025450693, -0.019545007, -0.00028144283, 0.03307184, 0.019629374, -0.025000736, -0.003771905, 0.0052799643, 0.024536718, 0.01532666, -0.00026957874, -0.0007412866, 0.026195934, 0.028305108, 0.0038949403, -0.0042253775, -0.0481454, 0.006724748, -0.024747634, 0.004513631, -0.0020458982, 0.002566161, -0.028487902, -0.0147079695, 0.011586393, -0.04536129, -0.011136436, 0.017618628, 0.010159185, 0.0006459344, 0.042380325, -0.024846064, 0.0025046435, 0.034224853, 0.03354992, -0.052616846, 0.0040531284, 0.067943506, -0.011593424, -0.007874248, -0.016662471, 0.003010845, 0.038499445, -0.028909737, -0.009934207, 0.0002366229, -0.016128147, 0.008837436, -0.0023763354, 0.016662471, -0.0012391394, 0.013428404, 0.005554157, 0.010869274, 0.008471847, 0.07435539, -0.02540851, 0.016915571, 0.027503623, 0.008211715, 0.006517346, 0.013034692, -0.004415203, -0.0075578718, 0.04983274, 0.016029717, -0.005483851, 0.007614116, 0.00081950176, -0.01763269, -0.040692985, 0.025239775, 0.03878067, 0.08419821, -0.007818003, 0.03464669, 0.009413945, 0.011656699, -0.0350404, -0.04693614, 0.004970619, -0.0006261609, -7.255776e-05, 0.018406054, 0.008106257, 0.015186049, 0.012212114, 0.025717854, 0.032903105, -0.01324561, 0.0070692464, 0.062206555, -0.0063521275, 0.022315055, -0.015157927, -0.013913514, 0.024410168, 0.011923861, -0.01033495, -0.0034151033, 0.0069005126, -0.0031479413, 0.008816345, 0.0073750764, -0.0066087437, 0.003545169, -0.014180676, -0.011649668, -0.0015142108, -0.028628515, 0.009568617, 0.021682303, 0.036671497, -0.036052804, 0.03855569, -0.007585994, -0.023004051, 0.069237135, 0.000108314845, 0.0014852097, -0.0074453824, -0.004446841, -0.021471385, -0.003367647, 0.018940378, 0.02760205, 0.020234004, 0.013737749, 0.012704255, 0.041902248, 0.022525972, 0.0022656037, -0.0381901, -0.024747634, 0.03790888, -0.045305047, 0.008232807, 0.04851099, 0.026688075, 0.030006507, 0.0044116876, -0.03658713, -0.0025679187, -0.016873388, 0.0038843944, -0.020683961, -0.019615313, 0.0104825925, 0.020360554, 0.008282021, 0.028909737, -0.018391993, -0.016240636, 0.0011538937, 0.052701212, 0.002061717, 0.033128083, -0.04283028, 0.026125629, 0.056497727, 0.016732777, 0.008978048, -0.0037472981, -0.019066928, -0.020866755, 0.017013999, 0.022666585, -0.011614515, -0.021583876, -0.0125566125, -0.015368843, 0.010306828, 0.00899914, -0.015607883, 0.020023087, -0.0080640735, 0.014679847, 0.008443724, -0.024986675, -0.0078039416, 0.036334027, 0.004422234, -0.002200571, -0.00013380068, -0.0054557286, -0.013119059, 0.026055323, -0.011073161, -0.0055647027, -0.041142944, -0.020712083, -0.0037824512, -0.010053727, 0.028234802, -0.021907281, -0.027911395, -0.037852634, -0.0145814195, 0.014026004, -0.005367846, 0.026336545, -0.0025521, 0.0026188905, -0.03172197, -0.009638922, -0.032115683, -0.003937124, 0.008570274, 0.021738548, -0.0024835519, 0.0016697623, 0.01884195, -0.008106257, 0.030062752, 0.01313312, 0.02443829, 0.019010684, -0.02370711, -0.008021889, -0.01567819, -0.052701212, -0.017618628, 0.02394615, 0.0067669316, 0.0064821932, -0.013822117, 0.034421712, 0.035237256, -0.041002333, -0.01070757, -0.01640937, 0.016142208, 0.019362213, -0.0041621025, 0.03391551, 0.025478816, 0.019263783, 0.01009591, -0.051745057, -0.014750153, 0.010960671, -0.022624401, -0.000988675, -0.016043779, -0.0060603586, -0.020782389, 0.018138891, 0.008239837, -0.01946064, 0.032031313, 0.023172786, -0.00656656, -0.004046098, 0.01750614, 0.007346954, -1.8661241e-05, -0.032762494, 0.012683163, -0.009463158, 0.0308783, 0.00426053, -0.030315854, -0.025085103, 0.015228232, -0.0058775633, 0.003912517, -0.0018894679, 0.00814844, -0.009069446, -0.01594535, -0.005666646, -0.0060427818, 0.011663729, 0.0078601865, -0.029725285, -0.0031461837, 0.0075649023, 0.02930345, -0.024269557, 0.009624861, -0.0037472981, 0.022582216, -0.055879034, -0.011684821, 0.0006389038, 0.015340721, 0.014054126, 0.014426746, 0.041564777, 0.007818003, -0.013632291, 0.011712943, -0.021485446, 0.0011486207, -0.030540831, 0.014623603, 0.03405612, -0.005216689, 0.0048300074, 0.029247206, -0.0026575585, -0.004179679, -0.027812967, -0.0022708767, -0.017970158, 0.020838633, -0.00473861, 0.011691852, 0.009828748, -0.016128147, 0.0019474701, -0.037543286, -0.014068187, -0.008816345, -0.014932948, 0.025563182, 0.0149751315, -0.018532604, 0.005751013, 0.020487105, 0.008071104, -0.012219145, -0.0075649023, -0.00838748, 0.0032410966, 0.069349624, -0.04052425, -0.006679049, 0.009582678, 0.0086616725, 0.00814141, -0.025675671, 0.008000798, -0.025985017, -0.009652983, -0.029753407, 0.024002394, 0.014363471, -0.044405133, 0.0065454682, -0.009174905, -0.01884195, 0.009786565, 0.03200319, 0.02882537, 0.007016517, 0.030006507, -0.022877501, -0.021682303, 0.0075227185, 0.012113687, 0.0007689695, 0.0010844667, 0.0022234204, 0.028122313, -0.014595481, -0.043252118, -0.011691852, -0.03419673, -0.0064962544, 0.014736092, -0.008591367, 0.0036699618, -0.018687276, -0.0040531284, -0.0014201768, 0.010763816, 0.0013612957, -0.011178619, 0.0029299934, 0.011403598, 0.0056139166, 0.015298538, -0.010032635, -0.018462298, -0.014736092, 0.029050348, -0.003638324, -0.016620288, 0.012542551, -0.022146322, 0.007677391, 0.027447376, -0.0068302066, 0.00741726, 0.018926317, 0.014609542, -0.008900712, -0.014047096, 0.019671557, -0.028417597, -0.002893083, 0.0040074294, -0.03405612, 0.011466873, -0.031693846, -0.022005709, 0.011494995, 0.010623204, 0.015537578, -0.011333292, 0.017084304, 0.005033894, 0.0025538576, -0.0019791077, -0.014004911, -0.0001812571, -0.049410902, 0.010735692, -0.018223258, -0.005265903, 0.012662072, -0.018504482, -0.050310817, 0.040889844, -0.01520011, -0.007125491, 0.0013068087, 0.015157927, 0.009526433, -0.007832064, -0.00016829447, -0.045108188, 0.019193478, -0.025760038, 0.012008227, -0.0054170606, 0.00047104873, 0.010960671, -0.0051709902, -0.012071502, -0.04193037, 0.033521798, 0.03464669, -0.006169332, 0.026364667, -0.0036137172, 0.013934606, -0.010018574, -0.0016618529, 0.01094661, -0.006773962, 0.020726144, -0.028234802, -0.048004787, 0.02955655, -0.008619489, 0.004365989, -0.023102479, 0.0046120593, 0.0041867094, -0.020937061, 0.004777278, -0.04631745, -0.0038246345, -0.020248065, 0.020937061, 0.024635145, -0.029809652, 0.009624861, 0.020430861, 0.0066579576, -0.010496654, -0.042886525, 0.033746775, 0.026336545, 0.024621084, -0.022483788, 0.00523075, 0.018335748, 0.011600454, 0.014201768, 0.005315117, -0.03537787, 0.01058102, 0.0039336085, -0.036502764, 0.007881278, 0.02359462, -0.003485409, -0.025985017, 0.010292767, -0.0016548224, -0.039343115, -0.01215587, 0.005705314, 0.012050411, 0.0018877103, 0.006359158, 0.0316376, -0.0005453092, -0.0096037695, 0.007065731, 0.019545007, 0.010377133, 0.0478923, -0.008802284, -0.009702198, 0.012416001, -0.027039604, -0.011579363, 0.008204685, -0.0015546366, -0.008640581, 0.0072063426, -0.021176102, 0.0024607023, -0.011762157, -0.02272283, 0.01971374, -7.903908e-05, -0.023411825, -0.023875844, 0.0039090016, 0.01155827, 0.000709649, -0.034477957, -0.020374615, -0.0350404, 0.012176962, 0.07716762, -0.021724487, 0.024860125, 0.034252975, 0.017435834, 0.009294424, -0.033634286, 0.0061728475, -0.0053362087, -0.010763816, 0.004815946, -0.019432519, 0.0015484849, -0.0017760998, 0.018574787, 0.008921804, -0.0079094, -0.02588659, -0.021063613, 0.0062712757, 0.024114883, 0.016845265, 0.007157129, 0.009083507, -0.0073258625, 0.008310144, 0.0037789359, 0.024367984, -0.014293166, -0.047976665, 0.039230626, -0.032650005, -0.017013999, 0.0048018848, -0.013639322, 0.010433378, -0.036024682, -0.009800626, 0.005729921, 0.012971417, 0.0040144604, -0.024325801, 0.016043779, 0.0061728475, 0.027939517, 0.012985478, 0.02491637, -0.016760899, 0.013301854, -0.03684023, 0.01946064, 0.014553296, 0.01737959, -0.043195873, 0.020248065, -0.009146783, 0.017351467, -0.03858381, -0.040805474, -0.011389537, -0.013407312, 0.013344037, -0.010735692, 0.025281958, 0.013941636, -0.054697897, 0.008992109, 0.008021889, 0.017182734, -0.017070243, 0.028909737, 0.0032727341, -0.01398382, -0.027166154, -0.009034293, -0.031946946, 0.01070054, -0.014806397, 0.020683961, 0.011459842, 0.014736092, 0.005874048, 0.015636006, -0.02758799, 0.016971815, -0.009188966, -0.026378728, 0.015354782, -0.013048753, 0.010834121, 0.0020054723, 0.016240636, 0.030765811, 0.03135638, -0.02516947, 0.006636866, 0.0049565574, 0.009463158, -0.01861697, 0.009463158, 0.01167779, -0.0012039865, -0.021597937, -0.030484587, 0.011389537, -0.011523117, -0.025225714, 0.0084577855, 0.008851498, 0.005181536, 0.03563097, 0.0026522856, 0.041367922, 0.0021249922, 0.00923818, 0.017365528, 0.014145523, -0.01957313, -0.011691852, 0.0041726483, -0.013358098, -0.00045215405, -0.04027115, 0.013182334, 0.0049178894, 0.014096309, 0.01617033, 0.027939517, -0.012514429, 0.035462238, -0.032509394, 0.0044925394, 0.022540033, 0.015748495, 0.012324603, 0.004657758, 0.033240575, 0.041958492, 0.016212514, 0.012500368, -0.0050936537, -0.0223713, 0.019291906, 0.020234004, -0.037177697, -0.015846923, -0.011853555, -0.05107012, 0.004671819, 0.009526433, 0.011656699, 0.006077935, -0.01362526, -0.03928687, -0.009772504, -0.00838748, 0.0052553574, -0.013779934, -0.026927114, 0.008485908, 0.023763355, 0.008317174, 0.015720373, -0.014201768, -0.033578042, -0.028023886, -0.0045030853, 0.013779934, 0.0068548135, 0.0024940977, -0.0047245487, -0.015762556, -0.038246345, -0.004450356, 0.021443263, 0.012148839, 0.019010684, 0.033043716, 0.0006481314, -0.03900565, -0.025394449, -0.012176962, -0.014412685, 0.026125629, -0.021654181, 0.02360868, -0.03298747, -0.010173246, -0.006893482, 0.036783986, -0.009013201, -0.01874352, 0.0036207477, -0.025703793], "content_hash": "f6e3fe41379649aec77dee82cd39e9ac"}, {"content": "Some heads show patterns that change based on the input. They might show one pattern for factual prompts and another for creative prompts. **What it means**: These heads are flexible and context-sensitive. They don't have a single fixed pattern but adapt to the input. ## Tips for Reading Attention Maps\n\n- **Start with one head at a time**: Double-click individual heads to isolate their patterns. Looking at all heads simultaneously is confusing.\n- **Compare heads across layers**: The same type of pattern (like Previous-Token) may appear in early layers but not late layers, or vice versa.\n- **Match patterns to categories**: Use the head categories as a guide. If a head is categorized as \"Previous-Token,\" look for the diagonal pattern to confirm.\n- **Hover for details**: The exact attention weight tells you more than just the visual thickness. Two lines might look similar but have meaningfully different weights.\n- **Context matters**: The same head can show different patterns for different prompts. Try multiple inputs to understand a head's full behavior.", "source_file": "interpreting_attention_maps.md", "chunk_index": 2, "embedding": [0.029645778, 0.042226423, 0.04595848, -0.00037057223, 0.0073775854, -0.014017789, 0.03747106, 0.0080886325, 0.012964385, 0.039939035, 0.031210836, -0.014950803, -0.019788934, -0.0032749556, -0.030970057, -0.0006762473, -0.0146874515, -0.019834079, -0.0043866723, 0.030413259, -0.0015161483, 0.024965659, -0.0033596042, 0.07560426, 0.052068222, -0.0070352294, -0.016929695, 0.043851674, 0.02844189, 0.03440114, -0.02594382, -0.013197639, -0.010391071, -0.014536966, -0.011722874, 0.009405387, -0.012031371, 0.051255595, 0.007129283, 0.0029965562, -0.004499537, 0.034130268, -0.025206437, 0.024213228, -0.011813166, -0.016327752, 0.032595307, -0.016147168, -0.06470906, 0.107326746, -0.018238926, 0.078734376, -0.012580645, 0.026440423, -0.02114331, 0.010285731, -0.0435808, 0.003237334, 0.030578794, 0.008758296, 0.05543911, -0.0028009242, 0.005101482, 0.013679194, 0.0077726115, -0.020827288, -0.076747954, -0.02802053, 0.02973607, 0.023235068, 0.092759684, 0.05480707, 0.02353604, 0.0036925548, -0.0056056106, 0.0031733776, 0.014138177, -0.015266824, -0.00743778, -0.0039690733, 0.021820497, 0.0066740625, 0.007516785, -0.045145858, -0.008690577, -0.017471446, -0.036447756, -0.055800278, -0.025417117, -0.054656584, -0.0070690885, 0.010699568, -0.008359508, 0.023129728, 0.00018704963, -0.012422635, -0.016357848, -0.0022723414, -0.036658436, 0.040811855, 0.038825437, -0.00097627915, -0.021805448, 0.015334543, 0.044844884, -0.0132202115, -0.019247184, -0.029013736, -0.0446643, -0.0110757835, -0.07692854, -0.017110279, -0.011008064, 0.042557493, 0.053542987, 0.012986958, -0.05841874, 0.0056206593, 0.058057573, -0.031632196, -0.010887676, -0.010150294, 0.018946212, 0.02376177, -0.00914956, -0.02540207, -0.0013261596, -0.018389411, 0.006933651, -0.0062827985, 0.014168275, -0.00031907775, 0.016147168, -0.052579872, -0.012369964, -0.034551628, -0.0015885698, 0.030488502, -0.050262388, 0.007840331, 0.031842876, -0.0026128164, -0.010977968, 0.037832227, -0.028803056, 0.015921438, -0.022843804, 0.07090909, -0.024258373, 0.043099243, -0.0067304946, 0.011459523, -0.026169548, -0.033919588, -0.020180197, -0.023806915, 0.00525573, -0.031872973, -0.04075166, -0.030037044, 0.013363173, -0.029043835, -0.0063655656, -0.009510727, -0.03518367, 0.020511268, -0.014010265, -0.040029325, -0.020857386, -0.0036342414, -0.02189574, -0.034912795, -0.018825822, -0.024107886, 0.031090446, -0.022091372, -0.052008025, -0.012264624, -0.028757911, -0.010172866, 0.0010327115, 0.011685252, -0.0007655985, 0.02397245, -0.024920512, -0.017606884, -0.012934288, 0.020902531, -0.027117612, -0.004401721, -0.010819957, 0.0022253145, 0.073798425, 0.0036775062, 0.0017870235, -0.07325668, 0.08397129, 0.012625791, 0.01449182, -0.0021538336, 0.022587977, 0.011895933, 0.03289628, 0.018344266, -0.009119463, 0.02550741, 0.019849129, -0.010187915, -0.028923444, -0.024228277, 0.024484104, -0.06657509, 0.00996971, 0.039367188, -0.06079642, 0.0066928733, -0.026004013, -0.016613675, -0.020420976, 0.07879457, 0.034160364, 0.015206629, 0.03780213, 0.044604108, -0.058328446, 0.031180738, 0.06747801, -0.027644312, -0.01831417, -0.034039974, 0.04622936, 0.03530406, 0.016523384, -0.0145821115, -0.032836087, -0.046018675, 0.017847663, -0.024047693, 0.022828754, -0.03397978, -0.00179831, 0.01927728, 0.010421168, 0.01765203, -0.011421902, 0.020481171, -0.029615682, -0.029073931, 0.03930699, -0.0029927942, 0.08300818, -0.0053196866, -0.053452693, 0.013385747, -0.009029171, 0.00017317668, -0.0076936064, 0.038193393, -0.01182069, -0.0010402357, 0.020797191, 0.04011962, 0.002187693, 0.01774232, -0.015454931, -0.038975924, 0.005146628, 0.0069900835, -0.063986726, -0.05318182, -0.041774966, 0.005470173, 0.05294104, 0.02046612, -0.059562434, -0.019954469, -0.031421516, -0.02647052, -0.010887676, 0.014469247, -0.028607424, -0.033137057, 0.04282837, 0.015199105, -0.026320035, 3.8062426e-05, -0.018870968, 0.007151856, -0.012994482, -0.012821423, -0.00024477518, -0.09119463, 0.061458558, 0.003068037, -0.0078252815, -0.010255634, -0.019292329, 0.0115949605, -0.015304445, -0.01777242, 0.012603219, -0.039818645, 0.006790689, 0.032444824, 0.028456938, 0.011655156, 0.027298195, -0.0041684676, 0.013927497, -0.023957402, -0.028351597, -0.002627865, 0.03397978, -0.030368112, -0.038945824, -0.044303134, 0.052910943, 0.02648557, 0.0024811411, -0.009864369, -0.053843956, -0.0020597796, 0.032444824, 0.038644854, 0.005473935, 0.0010806789, -0.030006945, 0.026711298, -0.004059365, -0.026139451, 0.008427227, -0.0027087515, -0.0077124173, 0.03563513, -0.042226423, -0.020240393, -0.015289397, 0.010752238, -0.005075147, 0.021098163, -0.05646242, -0.032083657, -0.026380228, 0.018359315, -0.033708908, 0.038163297, -0.06416731, -0.011361707, -0.0022949143, 0.0021764066, -0.019593302, 0.017832613, 0.003577809, -0.025311777, -0.0023099629, 0.052038122, -0.013701768, -0.02145933, -0.010609277, 0.027945286, -0.013212687, -0.033076864, 0.009510727, 0.038434174, -0.0015471862, -0.012859045, -0.026395278, -0.020436024, -0.01045879, -0.026876833, 0.010345926, -0.010240586, -0.027870042, -0.020752044, -0.036327366, 0.014935754, -0.014822889, -0.010541557, -0.0017550451, 0.035454545, -0.007900525, -0.0040029325, -0.0026617243, -0.0042211376, 0.03343803, 0.03500309, 0.022196712, -0.0031677342, -0.07349745, -0.034220558, 0.00015812807, -0.01274618, -0.021444282, 0.019668546, -0.02005981, 0.030473454, 0.0030266535, 0.04761383, 0.03527396, 0.026891882, 0.037019603, 0.014664879, -0.044965275, 0.019758837, 0.026440423, 0.029450146, -0.0133782225, -0.02004476, 0.036808923, 0.011828214, 0.077650875, 0.005078909, 0.020210296, 0.010067526, -0.016327752, 0.026109353, 0.023942353, 0.0010327115, -0.0013111109, 0.026921978, -0.008841064, -0.07897515, -0.038945824, 0.009495678, -0.03214385, -0.014153226, 0.06380615, -0.031963266, 0.037892424, -0.052068222, -0.039186604, 0.07650718, 0.02690693, 0.015966585, -0.00754312, -0.02440886, 0.032053556, 0.015454931, 0.017245717, -0.053422596, 0.02114331, -0.0030755615, -0.03801281, 0.004115797, -0.018901065, 0.02701227, -0.02418313, -0.07289551, 0.0076860823, -0.06771878, -0.019548155, 0.028893348, -0.024484104, 0.077169314, -0.008999074, 0.019262232, 0.03930699, -0.0063053714, 0.054536194, -0.036989506, 0.0009217279, -0.021384086, 0.007072851, 0.01971369, 0.014589636, 0.028171014, 0.019969517, -0.02526663, 0.07090909, 0.015831146, 0.004431818, -0.004860704, 0.03214385, -0.008961452, 0.027207904, -0.015018522, 0.0018199424, 0.0029570535, 0.00806606, -0.060405158, 0.02025544, -0.031632196, 0.00058219343, 0.01546998, 0.0132728815, 0.053452693, -0.042557493, -0.0067756404, 0.03587591, -0.0001695321, -0.00909689, -0.033287544, 0.0046650716, -0.018931162, 0.04616916, -0.014288664, 0.0067229704, -0.027870042, -0.0031338749, -0.038584657, 0.034160364, -0.016478237, -0.008585237, 0.04622936, -0.013603952, 0.025687993, 0.04011962, 0.014627257, -0.022708366, 0.015891341, -0.015951537, 0.0040292675, -0.024438957, 0.034340948, -0.03987884, 0.003622955, 0.026184596, 0.036959406, 0.010586703, -0.0039765975, -0.03629727, 0.0056733293, -0.051255595, 0.008540091, 0.010481363, -0.03473221, -0.010985492, -0.03527396, -0.049931318, -0.012460256, 0.011632582, -0.03575552, 0.013287931, 0.006008161, -0.039036117, -0.0132202115, -0.007607077, -0.038975924, -0.01950301, 0.010466314, 0.01981903, 0.0072910558, 0.024333617, 0.021173406, -0.0095032025, 0.01766708, 0.022603026, -0.027253048, 0.015236727, 0.044483718, -0.0028479511, -0.030789474, -0.0021274984, -0.01323526, 0.0028761672, -0.0073249154, -0.023355456, -0.016508335, 0.0073813475, -0.02299429, 0.029389953, -0.037561353, 0.005477697, -0.019864177, 0.0010853816, 0.037440963, 0.0077650873, -0.0032241666, 0.014363906, -0.042858467, 0.0011079545, -0.021504477, 0.020962726, -0.0588401, 0.019969517, 0.0022742224, -0.028396744, 0.011948603, 0.000103047765, 0.003455539, -0.012753705, -0.0042211376, -0.029239466, 0.002894978, -0.0070013697, -0.011391805, 0.027388487, -0.014281139, -0.035815712, 0.015846195, 0.015861245, 0.0022008605, 0.0012010679, 0.040149715, 0.0030322967, 0.004014219, -0.0245142, -0.023882158, -0.019728739, 0.013363173, -0.0043151914, 0.018825822, 0.0006122907, -0.0015471862, -0.0015415429, 0.011151027, -0.057034265, 0.009420435, 0.023806915, 0.023460796, -0.014604685, -0.015161484, -0.026756445, -0.007990817, -0.02756907, 0.03181278, -0.01569571, 0.013084774, -0.0004361748, 0.01623746, -0.0042211376, 0.008209022, 0.01291924, 0.01084253, 0.015846195, 0.012053943, 0.027493827, -0.007095424, -0.008525043, -0.028833153, -0.042527396, 0.00773499, 0.038313784, 0.010722141, -0.009593494, 0.020676803, -0.018359315, -0.010391071, 0.017019987, -0.014807841, 0.0046763583, -0.0030943723, -0.03235453, -0.020180197, 0.015184056, -0.0017061372, 0.023280213, -0.016959794, -0.03169239, -0.018600093, -0.011655156, -0.025326826, 0.054445904, -0.0003581101, -0.00026711298, -0.057244945, -0.02036078, -0.005515319, -0.016794259, 0.003796014, -0.022753512, -0.009856845, -0.031632196, -0.062361475, -0.022211762, -0.0049246605, 0.008201498, 0.01367167, -0.0064031873, 0.010518985, -0.094264545, 0.054415807, 0.03443124, 0.039698258, 0.002917551, 0.015116338, -0.008005865, -0.017576788, -0.029766168, 0.014679927, -0.0030699181, 0.026891882, -0.03530406, -0.03473221, 0.023280213, 0.022693317, 0.024363713, -0.015199105, 0.024469053, -0.012986958, -0.0022949143, -0.017757371, 0.033919588, -0.011098356, 0.006418236, 0.0021707632, 0.009337668, 0.048877914, -0.01291924, -0.027824895, -0.022738462, -0.0017616289, -0.006136074, -0.008984026, 0.0053535462, -0.023415651, 0.024754979, 0.04698179, 0.0062602255, -0.004209851, -0.048306067, 0.0015086241, 0.0029006214, 0.056342028, -0.004774174, 0.057816792, -0.022256907, -0.003624836, -0.0056244214, 0.0030021993, -0.011760496, 0.010308304, 0.0028253782, 0.0013289811, 0.030518599, -0.0007637174, -0.0066853487, -0.03780213, -0.010917773, 0.017847663, 0.003455539, 0.035574935, -0.0014503107, -0.029013736, -0.0066703004, 0.02973607, -0.037079796, 0.0150561435, -0.007938147, -0.029374903, 0.043279827, -0.02680159, 0.010902724, -0.010782335, 0.028893348, 0.013784535, -0.013754438, 0.036778823, -0.035785615, -0.034070075, 0.003478112, 0.015951537, 0.016222412, -0.019849129, 0.02594382, 0.006271512, -0.00050506927, 0.0037019602, 0.01927728, 0.03343803, 0.054536194, 0.015357115, -0.046289552, 0.002699346, 0.011880885, -0.0121743325, -0.004646261, 0.0020597796, 0.0035044472, 0.0020315635, -0.005477697, -0.0037113656, 0.060314864, -0.0045145857, 0.045025468, -0.035785615, 0.050262388, -0.006760592, -0.011369231, -0.009932089, 0.02039088, -0.010165342, -0.040149715, 0.0010308304, 0.0059479666, 0.012460256, -0.029510342, -0.039367188, 0.0135362325, -0.009939613, -0.0066289166, -0.02636518, -0.0070427535, -0.05044297, -0.012219478, 0.013995215, -0.01459716, -0.119726814, -0.0095032025, 0.021053018, -0.05655271, -0.03662834, 0.0023626331, 0.03759145, 0.0038505653, 0.011933555, -0.03202346, -0.009029171, 0.009006598, -0.02299429, -0.010940346, 0.033739004, -0.03443124, -0.03202346, -0.0053685945, -0.012302246, -0.0119636515, -0.019773886, -0.018585045, 0.009947137, 0.0144767715, -0.030759377, 0.011617534, -0.02973607, -0.01590639, 0.022347199, 0.016282605, -0.037531257, -0.0005140044, -0.002208385, 0.022467587, 0.012588169, 0.00893888, -0.0062940847, 0.015311969, 0.01721562, -0.009051744, 0.0012923002, -0.017260766, 0.000364929, -0.01302458, 0.01753164, 0.01094787, 0.0028629997, 0.0053159245, 0.00948063, 0.029510342, -0.027749654, 0.00010698628, 0.010481363, 0.02756907, -0.0002976805, 0.02659091, -0.02157972, 0.01557532, 0.0065800087, 0.026289936, -0.029811313, 0.01351366, 0.05023229, 0.0046349745, 0.009006598, -0.01829912, 0.0012132949, 0.012497878, -0.026094304, -0.0076033147, -0.00607588, 0.006162409, 0.012211954, 0.0190666, 0.02016515, -0.019518059, 0.020646704, 0.021534573, 0.0010007331, 0.014032837, 0.07415959, -0.029796265, 0.00637309, 0.031572003, 0.027854994, 0.0058501507, 0.016899599, -0.012467781, -0.0018321694, 0.035815712, -0.0283215, 1.9560266e-05, -0.0054325513, 0.018901065, -0.02212147, -0.01438648, 0.038644854, 0.036869116, 0.078493595, 0.0056959023, 0.013310503, 0.006764354, 0.0027764703, -0.049600247, -0.053121623, -0.010850054, 0.010684519, 0.0073625366, -0.0023231306, -0.01611707, 0.02484527, -0.002530049, 0.030970057, 0.0066439654, 0.010631849, 0.0016826236, 0.05306143, -0.017787468, 0.022738462, -0.021338942, 0.007975768, 0.016478237, -0.017260766, -0.022151567, -0.018178731, 0.013656622, 0.0030228912, 0.0021237363, 0.016944746, 0.0013694243, 0.023656428, -0.008788394, -0.0063354685, 0.0061661717, -0.0020033473, -0.017411252, 0.016764162, 0.016147168, -0.0067079216, 0.0067944513, -0.012640839, -0.0009903872, 0.057997376, -0.010646897, 0.016478237, -0.039517675, -0.020195246, -0.0019252826, -0.0008864577, 0.024454005, 0.020857386, -0.00081309566, 0.015786001, 0.004349051, 0.02756907, 0.029013736, 0.013385747, -0.019578254, -0.0072383857, 0.026982173, -0.020315636, -0.0031997126, 0.067839175, 0.023295261, 0.004379148, -0.00036681007, 0.0009330144, 0.027584119, -0.012610743, 0.009864369, -0.034250658, -0.003406631, 0.013122396, 0.006817024, 0.01590639, 0.03443124, -0.0097138835, -0.026831686, 0.0106017515, 0.03825359, -0.036146782, 0.053783763, -0.02952539, 0.022587977, 0.033377837, 0.017968051, -0.0037395817, 0.0017070777, -0.01993942, -0.025251582, 0.0073587745, 0.048667233, -0.010872627, 0.00032542637, -0.0031790207, -0.0115949605, 0.036658436, 0.00710671, -0.014732597, -0.004721504, -0.009247377, 0.033588517, -0.00032801286, -0.0038073005, 0.00093818735, 0.05652261, -0.011459523, -0.018976308, 0.0028216161, 0.02963073, -0.033618614, 0.036237072, -0.008931356, 0.008321886, -0.043881774, 0.015229202, -0.0019506773, -0.0123323435, 0.022136519, -0.011707826, -0.018600093, -0.027915187, -0.038313784, 0.00019586718, 0.0024905463, 0.0069675106, 0.033618614, 0.00963864, 0.0036756252, -0.0029307185, -0.014446674, -0.0028197349, 0.012753705, 0.03256521, -0.015334543, 0.015357115, -0.0038110628, 0.0045597316, 0.022061275, 0.013634048, 0.027463729, 0.021820497, -0.027659362, -0.0063016093, -0.02156467, -0.042015743, 0.00056902587, -0.0049171364, 0.008389605, -0.020691851, -0.026891882, 0.020932628, 0.04282837, -0.045928385, -0.020962726, -0.01351366, 0.010579179, 0.030909862, 0.0077500385, 0.039728355, 0.022347199, 0.018043295, -0.00012826597, -0.04400216, 0.0048531797, 0.0032674314, -0.03226424, -0.009029171, -0.016478237, -0.03497299, -0.017817564, 0.03093996, -0.00352702, 0.0069599864, 0.015981633, 0.02669625, 0.00017576317, -0.0015377807, 0.022587977, -0.005594324, -0.0028310213, -0.0033031718, 0.024378762, -0.019096697, 0.024258373, -0.005278303, -0.015545223, -0.012091565, 0.01580105, -0.021624865, 0.013152493, -0.020556413, 0.01831417, 0.0059968745, -0.011850787, -0.013340601, 0.005455124, 0.008622859, 0.027870042, -0.022633122, 0.015424835, 0.020767095, 0.018600093, -0.006297847, -0.011459523, 0.01611707, 0.028788008, -0.045115758, 0.0021820497, -0.005086433, 0.0056770914, 0.017606884, 0.011128454, 0.043099243, 0.016071925, 0.0058764857, -0.0023889681, -0.012490354, 0.00018928341, -0.028968591, 0.0037264142, 0.033528324, -0.006561198, 0.019156892, 0.03509338, -0.018946212, -0.0005629124, -0.031752586, 0.0054137404, -0.010646897, -0.0072835316, 0.0009438306, 0.002526287, 0.008494945, -0.017305912, -0.0024754978, -0.026575861, -0.0045409207, -0.0040518404, -0.01012772, 0.0031677342, 0.027057417, 0.0028799295, 0.01829912, 0.031361323, 0.011685252, 0.015770953, -0.022843804, -0.02540207, 0.010360974, 0.07506251, -0.028803056, -0.023581186, 0.018058343, 0.023927303, 0.010398596, -0.001131468, -0.0018763747, -0.030909862, -0.04475459, -0.039939035, 0.01030078, -0.0010524627, -0.00509772, 0.021504477, 0.0011333491, -0.03608659, -0.015981633, 0.034130268, 0.0076973685, 0.028833153, 0.050954625, -0.01950301, -0.015786001, 0.014303712, 0.034491435, -0.0016845048, 0.00054410164, -0.004525872, 0.0042850943, -0.024754979, -0.03530406, -0.00719324, -0.035936102, -0.016402995, -0.004153419, -0.0023005577, 0.015590369, -0.030548695, 0.012196906, -0.020752044, 0.02081224, -0.0005864259, -0.017080182, -0.012836472, 0.00710671, 0.014025313, 0.036237072, -0.012430159, -0.0174263, -0.026169548, 0.022843804, -0.013144968, 0.006444571, 0.018269023, -0.029073931, 0.002336298, 0.018359315, -0.0082993135, 0.008111206, 0.008284264, 0.02495061, -0.020405928, -0.012896666, -0.009510727, -0.029721022, -0.0035684037, 0.0038599707, -0.037410866, 0.019442815, -0.017351057, -0.023475844, 0.024423908, -0.013957594, 0.033076864, -0.0217904, 0.009337668, 0.0026165785, 0.0077199414, -0.0316021, -0.0013289811, -0.017110279, -0.057034265, -0.02472488, -0.020887483, 0.0022930333, 0.025672944, -0.029389953, -0.028366646, 0.037982713, -0.017080182, 0.02102292, 0.026109353, 0.04093224, 0.018269023, -0.011685252, 0.0033539608, -0.041714773, 0.0039841216, -0.011263891, 0.009992283, 0.00089069013, 0.003086848, 0.0108651025, -0.024800124, -0.014251042, -0.042226423, 0.029555487, 0.028607424, -0.018253975, 0.016056877, -0.013408319, 0.01513891, 0.011752971, -0.00020680093, 0.0027294431, 0.003380296, 0.013257833, -0.01720057, -0.04129341, 0.02865257, -0.009172133, -0.012979434, -0.012430159, -0.00050224765, 0.01733601, 0.0018453369, -0.0017117804, -0.034009878, -0.004228662, -0.0135362325, 0.0381332, 0.02799043, -0.003867495, -0.019096697, 0.0063354685, 0.018449606, -0.01405541, -0.023505943, 0.00032236963, 0.04595848, 0.04024001, -0.009269949, -0.018253975, 0.025281679, 0.025341874, 0.0067154462, 0.015755903, -0.011564864, 0.015635515, 0.0005615016, -0.022482637, 0.011098356, -0.008509994, -0.004913374, -0.025537506, -0.0008394308, 0.011188648, -0.0121743325, -0.016026778, 0.002772708, -0.00093818735, -0.011113405, 0.018028246, 0.02254283, 0.023520991, 0.00011750855, 0.019051552, 0.026530715, 0.0019318664, 0.039818645, -0.0059554907, -0.020330684, 0.010202964, -0.014416576, -0.0074791634, -0.002526287, 0.037290476, -0.0035853335, 0.040601175, -0.01105321, -0.0044619157, -0.007614601, -0.019021453, 0.011135978, 0.007520547, -0.02114331, -0.016929695, 0.017140377, 0.016719015, -0.0015782239, -0.034702115, -0.028998688, -0.015116338, 0.007633412, 0.06386634, -0.03521377, 0.025311777, 0.016372897, 0.003867495, 0.022512734, -0.027659362, -0.020285537, -0.01613212, -0.032595307, 0.026350131, -0.0021011634, -0.0036737442, 0.0017691533, -0.0016139643, -0.00550027, 0.0019807743, -0.03364871, -0.006162409, 0.020541364, 0.018449606, 0.028757911, 0.003621074, 0.011775544, -0.012701035, 0.004420532, 0.011888409, 0.027087513, -0.0022008605, -0.020405928, 0.014935754, -0.011256367, -0.026756445, -0.012053943, -0.008953928, 0.018359315, -0.048396356, -0.025296729, 0.014800317, 0.010503936, 0.009217279, -0.033919588, 0.023566136, 0.010391071, 0.01018039, 0.015259299, 0.028998688, -0.001787964, 0.02440886, -0.026861785, 0.023129728, 0.002432233, 0.015500077, -0.00021820498, 0.025116146, 0.010240586, 0.0035834524, -0.022136519, -0.019126795, -0.007592028, -0.02472488, 0.01155734, -0.007294818, 0.004770412, 0.029540438, -0.03975845, -0.010225536, 0.015041094, 0.012279673, 0.0021782876, 0.011956127, -0.008201498, -0.019232135, -0.029239466, 0.01012772, -0.008931356, 0.021805448, -0.008652956, 0.018539898, 0.005733524, -0.011181124, -0.005135341, 0.034762308, -0.030533647, 0.010609277, 0.00080274977, -0.042978857, 0.0007519607, -0.0056620426, -0.0060232095, 0.0123323435, 0.018765628, 0.0014540728, 0.039487574, -0.04039049, 0.014220945, 0.018675337, -0.0020785904, 0.008103681, 0.0032279287, -0.0062827985, 0.0059705395, -0.027283145, -0.036989506, 0.016929695, -0.034039974, -0.0033501987, 0.0014051648, -0.015424835, 0.014484296, 0.0068734568, -0.0049735685, 0.049841024, -0.0134835625, -0.018780677, 0.016944746, 0.0048983255, -0.003794133, -0.017591836, 0.021925837, 0.0030812046, -0.03479241, -0.01187336, 0.007953195, 0.003790371, 0.008005865, 0.018539898, 0.012475305, 0.007945671, 0.023400603, -0.035394352, 0.008449799, 0.018178731, 0.00012215246, -0.004303905, 0.024785075, 0.028502084, 0.041805062, -0.006147361, 0.013340601, -0.0017240074, -0.01960835, -0.0060608312, 0.011625058, -0.014235993, -0.011188648, -0.023581186, -0.049871124, -0.00893888, 0.01993942, 0.0067229704, -0.004499537, -0.021293795, -0.036778823, 0.011188648, -0.009382814, -0.0068471218, -0.027523924, -0.017802516, 0.013784535, 0.005789956, -0.001302646, 0.007441542, -0.026244791, -0.031993363, -0.021760304, -0.011459523, -0.015786001, 0.028848203, -0.009300047, -0.011451999, -0.023806915, -0.05733524, -0.0024736167, 0.0074264933, -0.018329218, 0.03584581, 0.049208984, -0.028381696, -0.017110279, -0.025025854, -0.025281679, -0.032956474, 0.021399137, -0.043610897, 0.016628724, -0.041714773, -0.016643772, -0.010669471, 0.027102562, -0.017381154, -0.022061275, -0.008449799, -0.011339135], "content_hash": "dfe19cc84fe7f544b03e606f288273e1"}, {"content": "# Interpreting Attribution Scores\n\n## Quick Reference\n\nToken attribution scores tell you how much each input token influenced a specific prediction. Here's how to read and interpret them.\n\n## Understanding the Scores\n\nAttribution scores are **normalized** so that the most influential token gets a score of 1.0, and all other scores are relative to it.\n\n### High Score (0.7 - 1.0)\n\nThis token was **highly influential**. The model relied heavily on this token when making its prediction. For factual predictions, these are usually the content words that carry the key information.\n\n**Example**: In \"The capital of France is\" \u2192 \"Paris\", the token \"France\" typically gets the highest score because it directly determines which capital the model predicts.\n\n### Medium Score (0.3 - 0.7)\n\nThis token had **moderate influence**. It contributed context that helped the prediction but wasn't the primary driver.\n\n**Example**: In the same prompt, \"capital\" might get a medium score -- it tells the model to predict a city name, but \"France\" specifies which one.\n\n### Low Score (0.0 - 0.3)\n\nThis token had **minimal influence** on this specific prediction. It may be a function word (the, of, is) or a word that doesn't directly relate to what's being predicted.\n\n## Comparing Attribution Methods\n\n### Integrated Gradients vs. Simple Gradient\n\n- **Integrated Gradients** averages gradients over many intermediate steps between a \"blank\" baseline and the actual input. This produces more reliable, less noisy scores. Use it when you want trustworthy results.\n- **Simple Gradient** takes a single gradient measurement. It's faster but can be noisy -- scores may overemphasize some tokens or miss subtle contributions. Good for quick exploration.\n\n**When they disagree**: If the two methods give very different rankings, the attribution is likely noisy. Trust Integrated Gradients for the more accurate picture.\n\n## Why Results Vary by Target Token", "source_file": "interpreting_attribution_scores.md", "chunk_index": 0, "embedding": [-0.0010772948, 0.007208415, 0.012402624, -0.02312064, 0.025464443, -0.018262131, -0.005282712, 0.0033783722, -0.025440028, 0.009936748, -0.030225292, 0.013367001, -0.0531262, -0.028613929, -0.015039402, -0.0029343315, -0.00014057478, -0.001954695, -0.015796255, 0.038770407, -0.04746201, -0.023962945, 0.05376098, 0.017529693, 0.04116304, -0.016345585, -0.030347366, -0.025635345, -0.009808571, 0.02185108, -0.0010849244, -0.014331379, -0.013061819, -0.056104783, 0.0024933424, -0.026538685, -0.025317956, 0.009637669, 0.03549885, -0.03237378, -0.012536905, 0.017432034, -0.04433694, 0.00023479993, -0.025415614, 0.0052918675, 0.049805813, -0.07900569, -0.025830662, 0.07910335, -0.05117303, 0.021741213, 0.0021439083, 0.013806464, -0.018323168, 0.028858075, -0.01099268, 0.004425149, -0.027979149, -0.053272687, -0.020752423, -0.02719788, 0.051124204, 0.010186998, 0.038453016, -0.015564317, -0.03217846, 0.0068055736, -0.0067689517, -0.037842654, 0.056788392, 0.031128634, 0.06772614, -0.0019058659, -0.038770407, -0.027735002, -0.009216517, 0.043726575, -0.029297538, -0.031299535, 0.023767628, -0.005862559, -0.008752639, -0.019348582, -0.0045014443, -0.019043399, -0.008398627, -0.053370345, -0.0042756093, -0.046900474, -0.041236285, -0.02443903, -0.004391579, 0.010852296, 0.01627234, -0.03576741, -0.046802815, -0.02130175, -0.001602209, -0.0074952864, 0.008740432, -0.026685173, 0.022546897, -0.017737217, 0.021509275, -0.041871063, 0.002853458, -0.022424823, 0.003668296, -0.015039402, -0.022607934, 0.036499847, -0.052930884, 0.02243703, -0.0017807409, -0.03540119, -0.077003695, 0.03962492, 0.004684554, -0.029004563, -0.013818672, -0.0489513, 0.003259351, 0.030762414, 0.014709805, -0.028369782, 0.011853295, -0.018457448, -0.025952736, -0.044898476, -0.003787317, -0.016760632, -0.025562102, -0.01627234, -0.009741431, -0.012213411, 0.032349363, 0.019043399, -0.04270116, 0.02357231, -0.031153047, -0.024085017, -0.01373322, -0.010205309, -0.063526824, -0.0018753476, -0.011950954, 0.031177463, -0.06089005, 0.03313063, 0.011206308, -0.017895913, -0.04318945, -0.016650766, -0.054298103, -0.019507278, 0.011407728, 0.03068917, -0.0015594835, -0.055567663, -0.0065309093, 0.039917894, 0.019165473, -0.050928887, -0.041846648, 0.01962935, -0.047291107, -0.01563756, -0.009381316, -0.032569095, -0.037256703, -0.043140624, -0.012854295, -0.036426604, 0.00489513, -0.032569095, -0.05571415, -0.028638342, 0.022998566, -0.015320171, -0.02683166, 0.008416939, -0.02058152, 0.022888701, 0.020825665, 0.000917074, -0.041553672, 0.015307963, -0.003100656, -0.0015747426, 0.035254702, 0.017346583, 0.07910335, 0.014941744, -0.038745992, -0.043995135, 0.0486095, 0.03969816, 0.0024429874, 0.016711803, -0.001076532, 0.0011291759, 0.037232287, -0.02452448, -0.011102546, 0.0023880545, 0.0017670078, -0.04189548, -0.07426926, -0.009411834, -0.0018738216, -0.043213867, 0.02751527, 0.004583844, -0.09536348, 0.010956058, -0.025415614, 0.0486095, 0.035645336, 0.0088136755, 0.007403732, 0.011596941, -0.009631566, -0.031616926, -0.050196446, 0.010193101, 0.050538253, -0.019372996, 0.059815805, -0.055421174, 0.045484427, 0.03386307, -0.019202095, -0.027100222, -0.016064817, -0.044825234, -0.0011383314, -0.01004051, 0.025952736, -0.046461012, -0.00073014956, -0.00022030374, 0.04714462, 0.049708154, 0.000374421, -0.007245037, -0.02384087, -0.038745992, 0.03535236, 0.0075135976, 0.030664755, -0.0062806597, -0.030152049, 0.03640219, 0.0059113884, -0.014856293, 0.013635562, 0.024597723, 0.014502281, 0.0026337265, -0.009662083, -0.010333486, -0.035742994, -0.02592832, -0.02570859, -0.014319171, -0.0390878, 0.029102221, -0.058741562, -0.034473434, -0.009295864, -0.016333377, 0.009185999, -0.01264677, -0.0027405405, -0.016394414, -0.04384865, 0.01359894, -0.033887483, -0.023059603, -0.0035645338, -0.008563426, 0.04633894, 0.0137820495, -0.028247708, -0.015369, 0.008252139, 0.052735567, 0.021387203, 0.009424041, -0.009759742, -0.026685173, 0.04238377, -0.034473434, -0.025586516, 0.0071168602, 0.01890912, -0.0020157315, -0.042994134, -0.027075807, 0.030200878, -0.038477432, 0.007946957, 0.010650875, 0.029297538, -0.012768843, -0.0004333594, 0.026441026, 0.016394414, -0.0140384035, -0.030005561, 0.009027303, 0.007220622, 0.033106215, -0.039478432, 0.053419176, 0.06513819, -0.03242261, 0.017712804, 0.022925323, -0.008117859, -0.05825327, 0.032837655, 0.0063111777, -0.007599049, 0.010498284, -0.0122988615, 0.0020477758, -0.0108767105, 0.016467657, 0.021362787, 0.009399626, -0.009051718, 0.060011122, -0.06460107, -0.0010933169, -0.021228507, 0.022949737, -0.0067689517, 0.048389766, -0.03540119, -0.04265233, 0.004455667, -0.014502281, 0.029346367, -0.014795256, 0.009497285, 0.03762292, 0.035718583, -0.0024048395, 0.017212303, 0.002851932, -0.0077943658, -0.007855402, 0.023230506, 0.041553672, -0.012335484, 0.012219515, -0.02624571, -0.016345585, -0.018371997, -0.009253139, 0.0243902, 0.02053269, 0.008435249, -0.061085366, 0.013501282, 0.016943742, -0.005743538, -0.024561102, -0.008075134, 0.043506842, -0.055128198, -0.031299535, -0.02793032, 0.009234828, -0.004150484, 0.018750424, -0.036109213, -0.0012932116, 0.03164134, -0.021741213, -0.008117859, -0.033325948, 0.048487425, 0.007916438, 0.017810462, -0.008123963, -0.018103436, 0.036060385, -0.03371658, -0.008593944, -0.009259243, 0.05004996, 0.013379209, 0.058643904, -0.009308072, 0.03994231, 0.053321518, 0.042359356, -0.026123637, 0.0077943658, -0.071925454, -0.010003888, 0.03440019, 0.075880624, -0.01767618, 0.023596724, 0.018396411, 0.0563001, 0.05263791, -0.034204874, 0.029224293, -0.0563001, 0.0074403537, 0.010199205, 0.050196446, 0.028223295, 0.045850646, 0.03747643, -0.020300752, -0.051710155, -0.0030258864, -0.050928887, -0.013586733, -0.012805466, 0.005359008, -0.012463661, 0.025317956, -0.015832877, 0.016931536, 0.015417829, 0.04121187, -0.03630453, -0.008679396, -0.007452561, 0.06567531, 0.0069520613, 0.0015488021, 0.006604153, 0.0155887315, -0.024182675, -0.041724578, -0.07715018, -0.038501848, -0.011926539, -0.03127512, -0.06460107, -1.353056e-05, 0.007336592, -0.0011665608, 0.039966725, -0.0013923959, 0.04079682, -0.03142161, -0.01940962, 0.044214867, 0.020544898, 0.033472437, -0.036158044, 0.057276685, 0.003436357, -0.00729997, 0.006091446, 0.008777054, -0.022266129, 0.0005157587, -0.02619688, -0.0013786628, 0.011725118, 0.008850298, -0.012939746, -0.015442244, -0.039234284, 0.0064210435, -0.025269127, 0.060450584, -0.03440019, -0.006219623, -0.010211413, -0.013721013, -0.0122988615, -0.037012555, -0.012817672, -0.013757635, 0.025781833, 0.023694383, 0.013171684, -0.01767618, -0.024402408, -0.012573526, -0.067481995, 0.014807464, -0.011761741, 0.012018094, -0.02507381, 0.019202095, -0.013147269, 0.0064576655, -0.033325948, 0.04316504, -0.034033973, -0.045337938, 0.08437691, -0.01949507, 0.01092554, 0.07280438, 0.005667242, -0.04316504, 0.034619924, -0.029712586, -0.01067529, -0.022754421, 0.0005947247, -0.043458015, 0.011065924, -0.018384205, 0.047974717, 0.010705808, 0.004269506, 0.011389418, -0.003958219, -0.02058152, -0.008392524, 0.0170414, -0.016113646, -0.027905904, -0.019641558, -0.032129634, -0.05000113, 0.007855402, -0.020776836, 0.016528694, 0.035034973, -0.040284112, 0.0029740052, 0.010577632, -0.02729554, -0.049610496, 0.010449455, 0.021631349, -0.014783049, 0.023535687, 0.029444024, 0.012524697, 0.006021254, -0.00228887, -0.03269117, -0.00096437725, 0.011035406, -0.033667754, -0.041431602, 0.038330946, -0.025586516, -0.01745645, -0.029028976, 0.022864286, -0.005316282, -0.0148196705, -0.025049394, 0.0139773665, -0.013367001, 0.010504388, 0.0093935225, -0.024683176, 0.011932643, 0.028223295, 0.026441026, 0.024902906, 0.014514488, 0.021741213, 0.0319099, 0.008764846, 0.03295973, 0.014380208, 0.0058015226, 0.0021179677, -0.0070008906, 0.03662192, -0.0126833925, -0.003436357, -0.0098451935, -0.007849298, -0.0068116775, 0.00054513256, -0.012500282, 0.019116644, 0.0027756365, 0.009185999, 0.032837655, 0.011676289, 0.012866502, 0.014404623, 0.018530693, -0.019824667, 0.054884054, 0.016308961, -0.023193883, -0.044507843, 0.013989574, 0.017102437, -0.03916104, -0.009570529, -0.0012764266, 0.01405061, 0.030176463, -0.013891916, 0.018640557, -0.0058411965, 0.0016296755, -0.026099222, -0.054200444, -0.0039368565, 0.010797363, -0.0095888395, 0.011889917, -0.02076463, 0.027686173, -0.009228724, -0.031226292, 0.0040924996, -0.016919328, -0.01309844, 0.014929537, 0.0040802923, 0.038208872, 0.007953061, -0.0074281464, 0.018555107, -0.0010284656, -0.0329109, 0.029004563, 0.009558322, 0.0047120205, 0.017065816, 7.55804e-05, -0.0058045746, -0.0030945523, 0.004071137, 0.0044526155, 0.02393853, 0.03630453, -0.011194101, -0.007104653, -0.0010505914, -0.02081346, 0.005545169, 0.008593944, -0.023230506, -0.03010322, -0.03386307, -0.031714585, 0.018542899, -0.007525805, -0.004138277, -0.028662758, 0.02434137, 0.008282658, -0.04206638, -0.0137820495, -0.0023193883, -0.015344585, -0.057911467, -0.020935532, -0.020544898, 0.011413832, -0.004354957, 0.026758417, 0.0053620595, 8.7024746e-05, -0.08598827, 0.022754421, 0.00092928123, 0.039746992, -0.0009956585, 0.0072816587, 0.010870607, -0.035059385, -0.024280334, 0.030298537, -0.0012382787, 0.013708806, -0.027026977, -0.0477794, -0.03264234, -0.026392197, 0.01117579, -0.0028595617, 0.02008102, -0.0571302, -0.022888701, -0.0022415668, -0.014929537, -0.006781159, -0.03762292, -0.006555324, 0.023438029, 0.055958297, -0.0037598505, -0.060987707, 0.016162476, -0.024011774, -0.03295973, -0.010962162, -0.029346367, 0.01867718, 0.009045615, 0.016247926, 0.00549634, -0.0681656, -0.041773405, 0.000987266, 0.01047387, -0.01054101, -0.010974369, 0.03396073, -0.0025864232, -0.0020355685, -0.0115236975, -0.0583021, -0.01727334, 0.0103640035, -0.04565533, 0.009564425, 0.010919436, -0.016199097, -0.022644555, -0.009124962, -0.023315957, 0.020715801, -0.0013130485, 0.0013290705, -0.018140059, -0.03364334, -0.0023178624, 0.007104653, -0.02008102, 0.03413163, -0.01428255, -0.009832986, 0.0024368837, -0.016540902, -0.008551219, -0.01899457, 0.003952116, 0.016516486, -0.025537686, 0.025562102, -0.017261133, -0.044092793, 0.0035736891, 0.0088747125, 0.025464443, -0.02012985, 0.031763412, 0.0022415668, 0.00472728, -0.03381424, 0.0024704537, 0.030908901, -0.002642882, 0.029785829, -0.023022981, 0.0032288327, -0.008484079, -0.0039917897, 0.019482862, 0.029248709, -0.024207091, 0.044922892, -0.017151266, -0.007470872, 0.012732222, -0.011664082, 0.028394196, -0.013183892, 0.032935314, -0.0030228344, -0.030200878, 0.0043976824, 0.02153369, -0.008050719, -0.01110865, 0.016150268, 0.022827664, 0.020630348, -0.0075441157, -0.044898476, -0.0030182567, -0.016601937, -0.026465442, 0.023047395, 0.019971155, -0.037281115, -0.024011774, 0.032031976, -0.01976363, -0.095998265, -0.0028259917, 0.052247275, -0.040186454, 0.003414994, -0.0028748207, 0.017151266, 0.0051331725, -0.043262698, 0.017163474, -0.023242712, -0.008844194, -0.009424041, -0.024121638, -0.019434033, -0.028418612, -0.023474652, -0.014514488, -0.0038147834, -0.0024368837, -0.012671185, 0.002026413, 0.009192103, -0.02135058, -0.030493854, 0.0026123638, -0.023792041, 0.00040665592, 0.004348853, 0.031128634, -0.011377211, 0.008624462, 0.029517269, -0.0036072594, 0.0021027087, 0.03671958, 0.003329543, 0.04084565, -0.0010528802, -0.023547895, 0.028247708, -0.032764412, 0.028613929, -0.011773948, 0.064161606, -0.058106784, 0.0026825557, 9.691934e-06, 0.0056153615, 0.017822668, -0.0063844216, 0.020984361, 0.0010895021, -0.0076906034, 0.03078683, 0.013244928, 0.02793032, -0.030493854, -0.013049611, 0.034888484, -0.0045868955, -0.0024124691, 0.006054824, 0.007312177, 0.019958947, 0.018872498, 0.0014290179, -0.0043213866, -0.027490856, -0.031177463, -0.009710913, 0.023108432, 0.0037049176, -0.018005777, -0.017554108, 0.002116442, 0.026441026, -0.0059571657, -0.058350928, 0.02474421, 0.069483995, -0.009729224, 0.012781051, 0.033032972, 0.00972312, 0.027026977, -0.011145271, 0.0067750555, -0.0069703725, 0.05141718, 0.041871063, -0.010638668, 0.020373996, -0.0042145727, -0.045264695, 0.0035004453, 0.04038177, 0.044751987, 0.020886702, -0.013220513, 0.017224511, 0.013574526, -0.039600503, -0.005081292, -0.02375542, -0.008172792, -0.0148196705, -0.02788149, -0.011822777, 0.007946957, 0.017810462, 0.047632914, 0.0036957623, 0.014343586, -0.023657762, 0.0017609041, 0.021948738, 0.01605261, 0.02108202, -0.0022659814, 0.0044068377, -0.0032562993, -0.0107485335, 0.0070863417, -0.018213302, 0.06323385, 0.0035828447, -0.00613112, 0.011633564, 0.004296972, 0.016211305, -0.0010200731, -0.002253774, -0.018799253, -0.02624571, 0.020251922, 0.016894912, -0.031397194, -0.037525263, 0.017773839, -0.02058152, 0.031543683, 0.040723577, -0.0057008127, -0.073585644, -0.021509275, -0.025244711, -0.053419176, 0.016577523, 0.014783049, 0.017883705, 0.013623355, -0.017602937, 0.011480972, -0.0065064947, 0.033570096, 0.0175419, -0.025220297, -0.0032990247, 0.034619924, -0.0052094683, 0.029517269, 0.022046397, 0.011297863, 0.06723785, -0.021692384, 0.0037049176, 0.004190158, 0.029639341, -0.016870499, -0.025244711, -0.03689048, 0.017651767, 0.031934317, 0.018213302, 0.015686389, -0.0002273611, 0.007220622, -0.0056153615, 0.029028976, -0.0022293595, 0.012347691, -0.009185999, 0.026270125, 0.025415614, -0.031616926, 0.012097442, -0.013830879, -0.038965724, -0.00067254633, 0.011126961, 0.03371658, 0.011596941, 0.04248143, -0.04538677, 0.0016067869, 0.020373996, -0.006836092, -0.000107576896, 0.029639341, -0.031348366, -0.011145271, 0.031348366, -0.005020255, 0.02026413, 0.028540684, -0.031153047, -0.006170794, 0.05263791, -0.0009811623, -0.0046601393, 0.005728279, -0.040772405, -0.00061913935, -0.028125636, 0.00954001, -0.03132395, -0.015979365, -0.016919328, 0.002676452, -0.039502844, -0.029785829, 0.015784048, 0.004684554, 0.009295864, -0.0035706374, 0.047217865, -0.026099222, 0.026123637, -0.00031491037, -0.03630453, 0.0047852644, -0.029395195, -0.00970481, -0.004617414, 0.0140384035, 0.012042508, 0.0056886054, 0.037744995, 0.008777054, 0.034717582, 0.016247926, -0.031104218, -0.022461446, -0.03169017, -0.031372778, 0.0037903688, 0.022266129, -0.0052979714, 0.0034302534, -7.295774e-05, 0.05117303, 0.0456065, -0.04968374, 0.020239715, -0.024097225, 0.025513273, 0.029566098, 0.02919988, 0.029639341, 0.00324104, 0.01092554, 0.013769843, -0.017493071, 0.0074647684, -0.029444024, -0.0019272286, 0.0053284895, 0.003399735, 0.023132848, -0.04411721, 0.01645545, 0.032764412, 0.034522265, 0.004336646, -0.024866285, 0.0045014443, -0.026367784, -0.0026856076, -0.0036896586, -0.025244711, 0.013537903, -0.03322829, -0.016492072, 0.020410618, -0.003964323, 0.021692384, -0.008197207, 0.0034851863, 0.015515488, -0.016931536, 0.021594726, 0.013879708, 0.007379317, -0.009912333, 0.013830879, 0.017505279, 0.0008171266, 0.003137278, -0.023022981, 0.018054606, -0.0017715854, -0.0038666646, -4.3154738e-05, 0.008184999, 0.023865286, -0.004300024, -0.010058821, -0.0024216245, -0.0018829771, -0.014392415, 0.010956058, -0.030860072, 0.05141718, 0.014795256, -0.025147052, -0.039380774, -0.04941518, -0.012036405, -0.009753639, 0.01212796, 0.009594943, -0.0109438505, 0.04487406, 0.012378209, -0.00032692694, 0.020251922, 0.007904232, -0.015369, 0.00507824, -0.0137820495, 0.007397628, 0.025366785, 0.0029633238, -0.05434693, 0.013855293, -0.008496285, -0.017761633, -0.009967267, -0.055518832, 0.0049805813, 0.0070863417, 0.02484187, -0.03269117, 0.046436597, -0.00977195, 0.008331487, 0.020740215, 0.0077089146, 0.029395195, 0.0706559, -0.013061819, 0.011981472, 0.004498393, -0.008252139, 0.023535687, -0.052784394, -0.004348853, -0.032886487, 0.0025665862, 0.005514651, 0.037647337, 0.027857075, -0.020007776, 0.0019485914, 0.008716017, 0.0023727953, 0.022998566, 0.033155046, 0.028174466, 0.00087816315, 0.029273123, -0.0064576655, 0.009662083, -0.013696599, 0.009936748, 0.0014870026, 0.019934533, 0.038770407, 0.006100602, -0.004846301, -0.028418612, 0.013696599, -0.010962162, -0.03603597, 0.014746427, 0.0016876602, -0.0013687443, -0.021094227, -0.00042420393, 0.014575525, 0.0048340936, 0.00044632968, 0.009167688, -0.015832877, -0.0001702347, 0.026489856, -0.015283548, -0.011474868, 0.0040131523, 0.002819888, 0.001744119, -0.017236717, -0.03747643, 0.01849407, -0.017932534, 0.001513706, -0.011023198, -0.0073426953, -0.02111864, 0.013830879, 0.023877492, -0.02044724, -0.011993679, 0.02103319, -0.027319953, 0.016846083, 0.004269506, -0.0051789503, 0.01201199, -0.024719797, -0.0085207, -0.022681177, 0.020862289, 0.018799253, -0.0126223555, -0.0025955788, 0.0021225456, 0.023376994, -0.027539685, -0.016870499, -0.016235719, -0.043238282, -0.001832622, -0.029395195, -0.017285546, 0.026074808, -0.0011177316, -0.029907903, 0.01759073, -0.023718799, 0.0034119424, 0.0059632696, 0.023633346, 0.0023315956, -0.01867718, 0.010785156, 0.013342586, 0.003050301, 0.019556107, 0.017151266, -0.02397515, -0.046973716, -0.0155887315, 0.010132065, 0.01099268, -0.03237378, 0.0131350625, 0.03762292, 0.0069886832, 0.04851184, -0.0038697163, 0.017016986, 0.007220622, -0.018005777, 0.022546897, -0.0105532175, 0.05620244, -0.009930644, -0.024646552, -0.0036347257, -0.012091338, -0.0037049176, -0.019665971, 0.011639668, 0.019067815, 0.024207091, 0.008612256, -0.036231287, -0.006659086, 0.02108202, 0.0017136007, 0.039283115, 0.011279552, -0.019421825, 0.005038566, -0.0034027868, 0.020837873, 0.012042508, 0.011548112, 0.009039511, 0.0063539036, -0.025635345, -0.038453016, 0.0060883947, -0.012481972, 0.027173465, 0.02729554, -0.0063111777, -0.014929537, -0.0011474869, -0.0130740255, 0.00743425, 0.00384225, -0.0031525372, -0.030225292, -0.027564099, 3.7027243e-05, -0.02425592, 0.0133303795, 0.026148051, -0.0088747125, -0.026587514, -0.005774056, 0.014941744, 0.00096590316, -0.014648769, 0.022058604, 0.02031296, -0.0019852133, -0.01984908, -0.011816673, 0.006872714, 0.030908901, -0.027954733, -0.012561319, 0.006591946, 0.0039338046, -0.00686661, 0.026270125, -0.027783832, -0.02968817, 0.0023743212, -0.009363005, -0.033936314, -0.0077028107, 0.009143273, -0.00908834, 0.016882706, -0.0059968396, -0.011444351, -0.0626479, -0.0036377776, -0.005090447, 0.026782831, 0.04250584, -0.015881706, 0.025098223, 0.006323385, 0.008960163, -0.013757635, 0.009381316, 0.030640341, -0.021008777, -0.012231722, 0.032666754, -0.027075807, -0.03535236, 0.008514597, 0.026270125, 0.009594943, -0.0042206766, 0.009942852, 0.01809123, 0.006958165, -0.010736327, -0.0077638472, -0.0097170165, 0.018970156, -0.013708806, 0.0008827409, -0.003451616, -1.252918e-05, -0.020776836, -0.022095226, -0.008435249, 0.008172792, -0.000748842, 0.01912885, -0.015857292, 0.003469927, -0.024158262, -0.014294757, 0.009460663, 0.019263132, 0.016846083, -0.04455667, -0.020117642, 0.004538066, 0.033521265, -0.008795365, 0.009643773, -0.03413163, 0.013696599, 0.0077821584, 0.024780834, 0.025903907, -0.01024193, 0.01062646, -0.01668739, 0.03549885, -0.017200096, -0.022888701, -0.011926539, -0.005389526, -0.045044962, -0.0020859237, 0.0051728464, 0.010864504, 0.0031281225, -0.080568224, -0.0009300442, 0.031519268, 0.0054536145, -0.0005443696, 0.014673184, -0.0014549585, -0.013830879, -0.03010322, -0.012066923, -0.0055207545, -0.0078431945, -0.0033020766, -0.017712804, 0.044214867, 0.037159044, 0.04221287, -0.019372996, -0.019739216, -0.0011474869, -0.00289008, 0.025317956, -0.01790812, -0.004153536, 0.015710805, 0.008654981, 0.011096442, 0.030664755, -0.013367001, -0.016553108, 0.00055962877, 0.038965724, 0.01428255, -0.032617923, -0.028711587, 0.012524697, 0.011346692, -0.027417611, -0.009051718, 0.016711803, 0.016235719, -0.0051820017, 0.026025979, -0.02166797, 0.020605935, 0.035059385, -0.018213302, 0.022656763, -0.035376776, 0.026343368, 0.021704592, -0.004669295, -0.025366785, -0.011474868, 0.014856293, -0.015784048, -0.0133303795, -0.008496285, -0.012817672, 0.0022003672, -0.0016113645, -0.006094498, 0.022925323, -0.01981246, -0.004498393, 0.018469656, 0.013171684, 0.042994134, -0.0039551677, 0.0024979203, 0.02312064, 0.024451237, 0.025171468, 0.01759073, 0.0033539576, -0.0035187562, 0.005322386, -0.006903232, 0.0047486424, -0.022339372, 1.42696745e-05, 0.0068910248, 0.008752639, -0.0015457503, 0.020044398, 0.028809246, -0.008587841, -0.008300968, -0.031934317, -0.023108432, 0.008374212, 0.024036188, 0.013354794, 0.0025864232, 0.0074769757, 0.024646552, 0.025220297, -0.015430036, -0.008539012, -0.029126635, 0.0053620595, -0.006622464, 0.00055581395, 0.026318954, 0.012793258, 0.024780834, -0.0067018117, -0.055958297, -0.0027832661, 0.013550111, 0.0041352254, 0.0073549026, 0.016931536, 1.1724499e-05, -0.009320279, 0.022192884, 0.0031952627, -0.0074220425, 0.008416939, -0.018555107, 0.04106538, -0.009350797, -0.019507278, 0.0089418525, 0.028174466, -0.020227509, -0.027735002, 0.0034973936, 0.0060029435], "content_hash": "dce30421164d4389bf4067fa798e236b"}, {"content": "faster but can be noisy -- scores may overemphasize some tokens or miss subtle contributions. Good for quick exploration. **When they disagree**: If the two methods give very different rankings, the attribution is likely noisy. Trust Integrated Gradients for the more accurate picture. ## Why Results Vary by Target Token\n\nAttribution is computed **with respect to a specific target token**. Different targets can be driven by entirely different input tokens.\n\n**Example** with prompt \"Alice gave Bob a gift because she\":\n- Target \"liked\": High attribution for \"Alice\" (she liked something) and \"Bob\" (she liked Bob)\n- Target \"was\": High attribution for \"she\" and \"Alice\" (describing Alice's state)\n- Target \"wanted\": High attribution for \"gift\" (she wanted to give a gift)\n\nThis is one of the most powerful uses of attribution -- it reveals which input tokens support different possible continuations.\n\n## Common Patterns\n\n- **Content words dominate**: Nouns, verbs, and adjectives typically have higher attribution than function words\n- **Recent tokens often matter more**: Tokens closer to the prediction point tend to have higher attribution, especially for local patterns\n- **Distant tokens can matter too**: For long-range dependencies (like pronoun resolution), distant tokens can have surprisingly high attribution\n- **Punctuation varies**: Commas and periods sometimes have notable attribution because they signal sentence structure\n\n## Tips\n\n- Try the same prompt with multiple target tokens to see how attribution shifts\n- Short prompts (5-10 tokens) give the clearest attribution results\n- If all scores are roughly equal, the model may be uncertain or the prediction may not depend on any single token\n- Use attribution alongside ablation for a fuller picture: attribution tells you which input tokens matter; ablation tells you which internal components matter", "source_file": "interpreting_attribution_scores.md", "chunk_index": 1, "embedding": [-0.0005734654, 0.0122070415, 0.02817837, -0.011655843, 0.03151245, -0.016159542, -0.0011914637, 0.044875666, -0.016199874, 0.024212426, 0.021308547, 0.0069370414, -0.028823676, -0.039525002, -0.04509077, -0.017772807, -0.0019812917, -0.026188675, -0.019883497, 0.0434775, -0.03834194, -0.023486456, 0.054743472, 0.019789388, 0.014721047, -0.025355155, -0.019762501, -0.025126608, 0.010842488, 0.010613942, 0.011131532, -0.015487348, -0.009625817, -0.0630249, -0.018807987, 0.02696842, 0.0018367699, 0.0035727108, 0.021335434, -0.027425513, 0.008288151, 0.014304287, -0.06028235, -0.03715888, -0.02617523, 0.02290837, 0.023029365, -0.044633675, -0.027089415, 0.09631194, -0.07528572, 0.058991738, -0.026659211, 0.022747042, -0.030141175, -0.0068059633, -0.0013628731, 0.0012578429, -0.015191583, -0.041380256, -0.022894925, -0.04460679, 0.027022196, 0.022195844, 0.028312808, -0.020300256, -0.05619541, -6.595903e-05, 0.0023577202, -0.007293304, 0.052108474, 0.041218933, 0.037615974, -0.0049339035, -0.037669748, -0.03788485, -0.0014712644, 0.031835105, 0.0062480425, -0.028823676, 0.03882592, -0.035626277, 0.0025543368, -0.030598268, -0.028743012, -0.015998214, -0.049097043, -0.038422603, -0.0390948, -0.050118778, -0.030571379, 0.003979388, 0.00033168567, -0.001934238, 0.025852578, -0.0203137, -0.033287045, -0.013578317, 0.022451278, 0.007831059, 0.0072462503, -0.019802833, -0.005878336, -0.030598268, 0.028017042, -0.028259033, -0.003764286, -0.021752195, -0.0022652936, 0.0022989032, -0.024588853, 0.024010766, -0.07023082, 0.0340399, -0.009141837, -0.0353574, -0.060228575, 0.0248174, 0.01726194, -0.033502147, -0.027586838, -0.015514236, -0.0029475703, 0.038557045, 0.033287045, -0.015500791, 0.017208165, -0.027358292, -0.0082209315, -0.009504822, -0.010923151, 0.010324898, -0.022343727, 0.004624694, -0.01757115, 0.005703565, 0.008691467, 0.016576301, -0.06496082, 0.024709849, -0.02270671, -0.029226992, 0.008570473, 0.013699312, -0.06194939, -0.0050212885, -0.022800818, 0.032668624, -0.051570717, 0.061196536, -0.0055119903, -0.018807987, -0.03861082, -0.0014595011, -0.05813133, -0.03379791, -0.017006505, 0.007911722, -0.0176787, -0.053641073, 0.03213087, 0.022330282, -0.0015401643, -0.02617523, -0.026161788, 0.028662348, -0.035599392, 0.008281429, -0.03261485, -0.03586827, -0.042133115, -0.051006075, -0.02620212, -0.02534171, 0.02817837, -0.04506388, -0.032803062, -0.029899186, 0.020824568, -0.010855932, -0.0038281444, 0.0070177047, -0.0016981299, -0.01431773, -0.01486893, -0.030571379, -0.05216225, 0.03213087, -0.00070286274, 0.0006478269, 0.03261485, 0.015796557, 0.076253675, 0.009195613, -0.028339695, -0.055872757, 0.07507062, 0.052646227, 0.004547392, -0.022195844, 0.014089185, 0.019856608, 0.045198318, -0.023755332, -0.016159542, 0.01827023, 0.00773023, -0.025597144, -0.08319072, -0.015971327, 0.01587722, -0.0577549, 0.028796788, 0.013753087, -0.08200766, -0.030410053, -0.024346864, 0.01719472, 0.029791635, 0.0351423, 0.009383827, 0.01820301, 0.007965498, -0.03740087, -0.04315485, 0.03826128, 0.051032964, -0.03968633, 0.044095922, -0.08722388, 0.03511541, 0.020784236, -0.026511328, -0.032749288, 0.010136684, -0.044875666, 0.0044768117, -0.017611481, 0.03640602, -0.039820768, 0.0066715246, -0.0050414545, 0.047161125, -0.012328036, -0.014129517, 0.0151378075, -0.047053576, -0.021537093, 0.026565103, 0.020044822, 0.039955206, -0.023029365, -0.0601748, -0.0028887533, 0.00024934195, -0.034927197, 0.015164695, 0.0134506, 0.014573164, 0.005938833, -0.000149038, 0.0150302565, -0.023930104, 0.01820301, -0.027358292, -0.02534171, -0.026807094, 0.027586838, -0.02554337, -0.026941532, -0.018592885, -0.014734491, 0.023029365, -0.00013821987, -0.025005614, -0.0034718816, -0.03032939, -0.005881697, -0.020125486, -0.029173216, -0.025435818, 0.0094981, 0.063401334, 0.009827475, -0.039632555, -0.015984772, 0.0028383387, 0.041622248, 0.0070647583, 0.017692143, -3.7101952e-05, -0.043047298, 0.044122808, -0.02308314, 0.00464486, -0.0017292189, 0.044848777, -0.0149227055, -0.021523649, -0.023419237, 0.030867144, -0.012045715, 0.012885957, 0.00032664422, 0.03619092, -0.0044902554, 0.0287699, 0.039874543, 0.027909491, -0.002145979, -0.018391225, 0.024871174, -0.0076428447, 0.032023318, -0.02679365, 0.0020233036, 0.04998434, -0.04288597, 0.026081124, 0.022881482, -0.0017199762, -0.06453062, 0.04267087, 0.0121331, 0.009518266, 0.0102240695, 0.0016922482, -0.0006780756, -0.019332297, -0.0012830502, -0.012959898, 0.021026226, -0.009961914, 0.04769888, -0.025516482, -0.0008822545, -0.03619092, 0.0040398855, -0.0024451055, 0.043101076, -0.039955206, -0.06361643, 0.017060282, -0.02055569, 0.013322883, 0.009820753, 0.009531709, 0.040600512, 0.049419697, -0.008751965, 0.0076630106, -0.010855932, -0.0016031825, 0.008052883, 0.003771008, 0.024629185, -0.020985894, -0.014828598, 0.024185536, 0.0026870952, 0.0015275606, 0.013793419, 0.018767655, 0.034228116, -0.010291289, -0.011662564, 0.011823891, 0.0064765885, -0.006382481, -0.005101952, -0.031700667, 0.021577425, -0.03643291, 0.005559044, -0.049581025, 0.02828592, -0.0053372197, -0.00096795923, -0.00585817, 0.029226992, 0.02301592, -0.05549633, -0.019493625, -0.008906569, 0.04038541, 0.0052666394, 0.013235498, -0.023526788, -0.043396838, 0.016240206, -0.0055892924, -0.014344619, -0.051006075, 0.047618218, 0.004214656, 0.030759593, 0.0087586865, 0.06184184, 0.048613064, 0.064853266, -0.020300256, 0.017100614, -0.0628098, 0.009202335, 0.03299128, 0.06103521, -0.0066816076, 0.034362555, 0.030383166, 0.02901189, 0.042348217, -0.019399516, 0.039740104, -0.009908139, 0.024844287, 0.00021783284, 0.022451278, 0.00859736, 0.05431327, 0.04113827, -0.015957884, -0.054609034, -0.028312808, -0.08200766, -0.017907247, -0.0036735397, 0.017167833, -0.015581455, 0.030248726, -0.012556583, 0.00091502396, 0.028420359, 0.05487791, -0.037562195, -0.019265078, -0.03296439, 0.061895616, 0.045359645, 0.0044600065, -0.012321315, 0.02751962, -0.040815614, -0.050145667, -0.053130206, -0.06824113, -0.025556812, -0.04132648, -0.054716583, 0.01033162, 0.0060228575, -0.020233037, 0.026981864, -0.00030563818, 0.039740104, -0.012079325, -0.0065942225, 0.05834643, 0.014546276, 0.04350439, -0.004839796, 0.05237735, -0.031136023, -0.026296226, 0.0038953638, 0.012368368, 0.013517819, -0.0019476819, -0.016307425, 0.029845411, 0.012724631, -0.00055456, -0.018445002, -0.005125479, -0.05286133, 0.0070647583, -0.017087169, 0.05033388, -0.02971097, 0.003434911, -0.019735614, -0.015783112, 0.01001569, -0.009067896, 0.022034517, -0.03188888, 0.064100415, 0.012717909, 0.012381812, -0.007817616, -0.03264174, 0.003160992, -0.07297337, 0.017060282, -0.015231914, 0.0029895825, -0.011003815, 0.032292195, 0.0059018624, 0.0050044837, -0.034174338, 0.046569593, -0.033851687, -0.059529494, 0.045010105, -0.02415865, 0.012227207, 0.07302715, 0.0011284455, -0.031189797, 0.042617094, -0.050145667, -0.034712095, -0.02156398, 0.011676009, -0.03498097, 0.0075957915, -0.027304517, 0.046650257, 0.036325358, 0.03586827, 0.0024635908, 0.0033374429, -0.026833981, -0.031566225, -0.0009335093, -0.02270671, -0.02890434, -0.0008242778, -0.04135337, -0.02887745, 0.0040298025, -0.04186424, 0.012227207, 0.002752634, -0.05323776, -0.0031492286, 0.012865792, -0.023661226, -0.051920258, 0.0022921814, 0.027775053, -0.010210626, -0.00050456554, 0.032776177, -0.0040701344, 0.006950485, 0.0017930773, -0.024104875, 0.0044331187, 0.009067896, -0.00488685, -0.04786021, 0.014842042, -0.024346864, 0.0064597838, -0.014102628, 0.018834874, -0.00051254785, -0.016966175, -0.03861082, 0.030248726, -0.016334312, 0.010042577, -0.007226085, -0.027049083, 0.03369036, 0.025328267, 0.004564197, 0.031942654, 0.011635677, 0.026645767, 0.0030685652, 0.04786021, 0.023029365, 0.035088524, 0.013201889, 0.007400855, 0.011911277, 0.019533955, -0.015675562, -0.011010536, -0.002120772, -0.0052464735, 0.011642398, 0.010177016, -0.020717017, 0.023110026, 0.011978496, -0.00932333, 0.03836883, 0.019211302, -0.0026736513, 0.019870052, 0.025395487, -0.026161788, 0.038557045, 0.0052095028, -0.015352909, -0.016804848, -0.0029274046, 0.02516694, -0.03344837, -0.02769439, -0.0054044393, 0.009067896, 0.028124593, -0.023042807, 0.0046818308, -0.0008940179, 0.009222501, -0.02038092, -0.036997553, -0.007199197, 0.0028148121, -0.021254772, 0.005733814, -0.0006150574, 0.013013674, 0.0010334982, -0.035626277, -0.0027055806, -0.006842934, -0.016011659, 0.011931442, -0.0028433802, 0.038126837, -0.008846072, -0.020233037, 0.029415205, -0.016791403, -0.039632555, 0.030302502, 0.012912845, 0.009195613, 0.025032502, 0.011198751, -0.019251633, 0.013322883, 0.014842042, 0.009330052, 0.0025845857, 0.048344187, -0.02284115, -0.001924155, -0.022168955, -0.0115617355, 0.009195613, 0.0047255233, -0.003255099, -0.017813139, -0.040197197, -0.03320638, 0.038799033, -0.01917097, 0.007548738, -0.018512221, 0.010620664, 0.014438725, -0.031378012, -0.026591992, -0.016643522, -0.011978496, -0.040439185, -0.022397501, -0.031431787, 0.016334312, -0.0020333866, 0.032803062, -0.0041272705, -0.0020804403, -0.08157746, 0.02936143, -0.004339012, 0.013497654, -0.0006028739, -0.00776384, 0.022168955, -0.041649137, -0.0269012, 0.015487348, -0.010990371, -0.016589746, -0.03546495, -0.049823012, -0.009296442, -0.01438495, 0.024548521, -0.0053271367, 0.006735383, -0.029657196, -0.007844503, -0.015662119, 0.01677796, 0.0077503957, -0.019601176, -0.0066412757, 0.022021072, 0.044257246, -0.019937271, -0.060067248, -0.012509529, -0.01462694, -0.01390097, -0.038745258, -0.015810002, 0.0020132207, -0.0010292969, 0.036997553, 0.0012469197, -0.03713199, -0.035814494, 0.017947577, 0.0077840057, 0.0057270923, -0.036298472, 0.035034746, 6.0064203e-06, 0.004181046, -0.015097476, -0.06802602, -0.026229007, 0.0051322007, -0.031566225, 0.019802833, 0.0018367699, -0.016912399, -0.007609235, -0.008167156, -0.028662348, 0.007878113, 0.008281429, 0.014358062, -0.024131762, -0.045117658, -0.03406679, 0.0067084953, -0.013410268, 0.043343063, -0.029334543, -0.008341926, 0.026188675, -0.0038550321, 0.010143407, -0.010519835, 0.004322207, 0.019856608, -0.020596022, 0.025691252, -0.008799018, -0.034496993, -0.012570026, 0.012025549, 0.024669517, 0.004974235, 0.03105536, 0.005757341, 0.015097476, -0.044311024, -0.005844726, 0.032050207, 0.009524988, 0.0018014797, -0.016724184, -0.005653151, 0.0025778636, -0.014761378, -0.0018838234, 0.02200763, -0.022652935, 0.051893372, -0.0013242221, -0.0040230807, 0.046919137, -0.030544491, 0.042993523, -0.0072798603, 0.011198751, 0.0061035207, -0.038046177, -0.00017918169, 0.015608343, 0.00044238762, -0.03299128, -0.0004415474, 0.014290843, 0.01750393, -0.01611921, -0.038073063, 0.0015157973, -0.025261048, -0.017772807, 0.0046885526, -0.013954746, -0.018458445, -0.010627386, 0.03861082, -0.03643291, -0.101635724, -0.013248942, 0.0577549, -0.04708046, -0.01178356, 0.004977596, 0.059744596, -0.00035941368, -0.042697757, 0.0011847417, -0.006846295, -0.0043524555, -0.024037655, -0.033098828, -0.008644414, -0.013511098, -0.016979618, -0.011279414, 0.003989471, -0.008489809, -0.0096863145, -0.019291965, 0.0018552552, -0.03317949, -0.030571379, -0.0012469197, -0.019923829, -0.0050448156, -0.0015368033, 0.021375766, -0.017974466, -0.006530364, 0.01965495, -0.008926735, 0.016656965, 0.02301592, 0.016509082, 0.0163612, -0.002645083, -0.020340588, 0.028312808, -0.03186199, 0.039471228, -0.014976481, 0.06501459, -0.05165138, 0.016011659, -0.0028652265, -0.0068967096, 0.007192475, -0.024319977, 0.006479949, -0.017584592, 0.0009419117, 0.00419449, 0.018619772, 0.03748153, -0.017208165, -0.022343727, 0.023594007, -0.014640383, 0.020811124, 0.007636123, -0.0049540694, 0.0043356507, 0.00045079004, -0.0119919395, 0.0050448156, -0.016751073, -0.009612373, 0.002769439, 0.026686098, -0.011756672, -0.010062743, -0.01618643, 0.008442756, 0.03188888, 0.013632093, -0.06028235, 0.040708065, 0.06372398, -0.0055926535, 0.01802824, 0.057701126, 0.0049238205, 0.042993523, -0.0021140499, -0.0015031936, -0.016630078, 0.049446587, 0.044122808, -0.039847657, 0.009182169, 0.0075823474, -0.045897402, -0.014546276, 0.023997324, 0.051140513, 0.023325128, -0.011037424, 0.024857732, 0.020569135, -0.051463168, -0.014479057, -0.006295096, -0.0076764547, -0.014465613, -0.022585716, 0.004779299, 0.014331174, 0.014425281, 0.037078217, -0.020300256, 0.017786251, -0.027291073, 0.03264174, 0.011050868, 0.0031693943, 0.012005383, -0.011205473, 0.0031828382, 0.007078202, 0.0031425066, -0.016213316, -0.023822553, 0.043585055, -0.015635232, 0.005101952, 0.0034080232, 0.017517373, 0.03740087, 0.005881697, 0.011306302, -0.026027348, -0.024373751, -0.00043860654, 0.029442094, -0.01882143, -0.0366749, 0.01695273, -0.0049103764, 0.014613496, 0.037965514, -0.009034286, -0.052216023, -0.035948932, -0.012072603, -0.04783332, 0.028608574, -0.011709618, 0.033421483, 0.019977603, 0.0030097482, -0.007286582, -0.015904108, 0.011514682, 0.0132422205, -0.004997762, 0.019372629, 0.043047298, 0.012159988, 0.026847426, 0.02290837, -0.008093215, 0.059905924, -0.037562195, -0.0016561177, -0.0039054467, -0.0018939064, -0.013175, -0.030732706, -0.014102628, 0.007615957, 0.0351423, 0.024682961, 0.028232144, -0.0034063426, 0.0010721493, 0.014210179, 0.029307654, 0.00013790479, 0.0025610588, -0.032561075, 0.009632539, 0.034362555, -0.026941532, 0.0020804403, -0.009121671, -0.017705588, 0.009645983, 0.012677577, 0.02253194, 0.023956992, 0.036029596, -0.03511541, 0.0021829498, 0.011460906, -0.011770115, 0.019090308, 0.004197851, -0.07275827, -0.00054405694, 0.03549184, -0.002169506, 0.015191583, 0.035384286, -0.012664134, -0.0048330743, 0.020152373, -0.00020228836, -0.026255894, 0.02360745, -0.041057605, 0.0075957915, -0.0070244265, 0.00038062982, -0.02925388, -0.0128120165, 0.0028416999, -0.012798572, -0.042025562, -0.018485332, 0.0034853255, 0.019883497, -0.005458215, 0.0107416585, 0.051974036, -0.01223393, 0.00946449, 0.0025106443, -0.028017042, -0.021496762, -0.025745027, -0.0058985017, -0.018861761, 0.037373982, -0.01948018, 0.011373521, 0.02145643, 0.005306971, 0.044391688, 0.016280537, -0.015312578, -0.008913292, -0.03498097, -0.036325358, 0.007743674, 0.021725308, -0.011393687, 0.012462475, -0.0035693496, 0.041891124, 0.06157296, -0.039471228, -0.00478266, -0.017450154, 0.01611921, 0.030490717, 0.021846302, 0.03043694, 0.012959898, 0.018498776, -0.011010536, -0.017961022, -0.00901412, -0.017786251, -0.0128321815, -0.0041306317, 0.025570257, 0.0040466073, -0.05127495, 0.024131762, 0.021268215, 0.025408931, 0.010002245, -0.011850779, -0.003982749, -0.026484441, 0.002665249, -0.006990817, -0.014721047, 0.02000449, -0.053883065, -0.014479057, 0.002374525, -0.011951608, -0.0030147897, -0.013833751, -0.019345742, -0.00596236, 0.0045507527, 0.01327583, 0.02367467, 0.006869822, 0.005831282, 0.0015620106, 0.024212426, -0.0023560396, 0.017436711, -0.014425281, 0.0012292747, 0.02194041, 0.019386074, -0.0171275, 0.015715893, 0.026161788, 0.0028030486, -0.015729338, 0.0071118115, -0.0052464735, -0.035249848, 0.0037306764, -0.031727552, 0.023661226, 0.011158419, -0.001136848, -0.024494747, -0.052834444, -0.027223853, 0.0007284902, 0.012838904, 0.007293304, -0.0010057702, 0.027170079, 0.017665256, 0.008644414, -0.005851448, 0.004204573, -0.016065435, -0.01785347, -0.031243574, -0.00028652267, 0.025314823, -0.0055691265, -0.032776177, 0.021348879, -0.022545384, -0.012086047, -0.0049137375, -0.039578777, 0.030705819, -7.914033e-05, 0.013295996, -0.025207272, 0.054824136, 0.00932333, 0.00064866716, 0.0034954085, 0.019265078, 0.00627493, 0.073995106, -0.02336546, -0.01989694, -0.0016998104, -0.018243343, 0.049339034, -0.005323776, -0.00046927537, -0.032157756, 0.028447246, -0.010405562, 0.025382042, 0.019036531, -0.010472781, -0.0017964382, 0.002166145, -0.0052095028, 0.02973786, 0.031969544, 0.021147221, 0.013679146, 0.052242912, -0.012368368, 0.007925167, -0.00092006545, 0.015823444, 0.002811451, 0.010197182, 0.025946686, 0.0059321113, -0.010479503, -0.015514236, -0.010304733, -0.00478266, -0.041165154, 0.014546276, -0.013302717, 0.017033394, -0.013860638, 0.0039995536, 0.011615511, 0.030598268, 0.0027610366, -9.542003e-05, -0.002332513, 0.0038449492, 0.025019057, -0.033475257, -0.008536863, 0.0052464735, 0.0006537086, -0.0014981522, -0.027183522, -0.026995309, 0.012227207, -0.013652259, 0.0021762277, -0.016576301, -0.0019207941, -0.012892679, 0.003539101, 0.03081337, -0.022047961, -0.013074172, 0.021174109, -0.026282782, -0.007306748, 0.0043524555, -0.0025526565, 0.009538432, -0.030840257, -0.011709618, 0.006617749, -0.01362537, 0.030383166, 0.010728215, -0.0063858423, 0.0054078, 0.023499899, -0.004712079, -0.016092323, -0.0076898984, -0.034201227, 0.004880128, -0.030732706, -0.007918444, 0.03500786, -0.010950039, -0.017624924, 0.036513574, -0.024682961, 0.012119656, 0.02655166, 0.016159542, 0.0022989032, -0.02128166, -0.023661226, 0.0126843, -0.0005587612, 0.0009217459, 0.011770115, -0.0039491393, -0.024938395, 0.012092768, -0.0009469532, -0.0110441465, -0.05466281, 0.0118843885, 0.03282995, 0.014613496, 0.044122808, -0.0106341075, 0.031673778, 0.011077756, -0.0074949623, 0.02738518, 0.004207934, 0.033018164, -0.03551873, -0.010163572, -0.01757115, -0.012301149, -0.0017174555, -0.01105759, 0.022679823, 0.01792069, 0.020838011, 0.009612373, -0.02887745, -0.0014242108, 0.0072193625, 0.008704911, 0.026860869, 0.016804848, -0.0115617355, 0.011050868, -0.0007020225, -0.004977596, 0.0051557273, -0.0031391457, 0.02637689, 0.014250511, -0.030302502, -0.018149236, -0.0046280553, -0.030517604, 0.035196073, 0.005528795, 0.023499899, -0.0066513587, 0.006207711, -0.015984772, 0.018565996, 0.002038428, 0.001167937, -0.0075352937, -0.033098828, -0.0036600959, 0.006527003, 0.013336328, 0.039820768, -0.0063589546, -0.014169848, 0.0030752872, 0.02003138, -0.00869819, -0.029092552, 0.012469198, 0.0073672454, -7.152563e-05, -0.009282998, -0.004449924, -0.008536863, 0.013322883, -0.02128166, -0.015756225, 0.011413853, 0.032426633, -0.005784229, 0.029146329, -0.02721041, -0.032534186, -0.0015527679, -0.016576301, -0.032749288, -0.0009368703, 0.005518712, -0.0038920029, 0.002759356, 0.0040197195, -0.025449261, -0.05022633, 0.0013948024, 0.0051691714, 0.019991048, 0.034093674, -0.029307654, 0.0300874, -0.0043423725, 0.010298011, -0.010681162, -0.011299579, 0.03715888, -0.002263613, 0.0011721381, 0.031673778, 0.0063757594, -0.030383166, 0.0024972004, 0.028958114, 0.025422374, 0.0023308324, 0.0059556384, 0.02020615, 0.022599159, -0.01414296, 0.014546276, 0.008039439, 0.014613496, -0.0009234264, 0.0030702457, -0.009666149, 0.013685868, -0.023217577, -0.020286813, -0.0016493958, 0.011245804, 0.011104643, 0.014895817, 0.00020165817, 0.011023981, -0.045359645, -0.009343496, 0.030974695, 0.011481072, 0.022397501, -0.037347093, -0.019802833, 0.0058346433, 0.027909491, -0.017987909, 0.014008521, -0.014371506, 0.022249619, -0.013369937, 0.03164689, 0.025382042, -0.016630078, 0.0028904339, -0.014304287, 0.02828592, 0.0053641074, -0.022437833, -0.02000449, -0.015621787, -0.05237735, 0.0059926086, 0.004594445, 0.0076630106, -0.0024249395, -0.05216225, -0.0021174108, -0.0008360412, 0.005632985, 0.009262832, 0.027559951, 0.002319069, -0.0034651596, -0.0010116518, -0.0003722274, 0.0003373573, 0.0129195675, -0.026470996, -0.016132655, 0.024481302, 0.012691021, 0.019601176, -0.0016804849, -0.028743012, -0.012462475, 0.0002974458, 0.014842042, -0.020058267, -0.008926735, 0.012180154, 0.0043524555, 0.0008053723, 0.022599159, -0.027492732, -0.016522527, -0.010284567, 0.035922043, 0.017840026, -0.0074680746, -0.01726194, 0.017033394, -0.009383827, -0.022720154, -0.019144082, -0.002592988, 0.009699758, -0.011373521, 0.042348217, -0.028662348, 0.029334543, 0.026108012, -0.026766762, 0.050495207, -0.021792527, -0.002655166, 0.010304733, -0.004359178, -0.027317962, 0.0032349331, 0.018619772, -0.0006381641, -0.018606327, -0.002325791, -0.017436711, -0.0007385731, -0.015366353, 0.00088897644, -0.0048969327, -0.026484441, -0.022693267, 0.017772807, -0.00221824, 0.037105106, 0.001577135, 0.0041306317, 0.02263949, 0.03640602, 0.03691689, 0.009081339, 1.5853011e-06, -0.009545154, -0.03221153, 0.0001500883, 0.0010595457, -0.012227207, -0.00095367513, 0.010311455, 0.017732475, 0.001410767, 0.013390103, 0.04315485, -0.025529925, -0.00081881625, -0.01618643, 0.008462922, 0.0163612, 0.026363445, 0.018391225, -0.01282546, -0.00036865636, 0.014963036, 0.043235514, -0.007199197, 0.011306302, -0.03476587, -0.004322207, -0.026121456, 0.0045104213, 0.028151482, 0.00478266, 0.02228995, -0.023903215, -0.038449492, -0.018552553, 0.004997762, -0.0062413206, 0.005390995, 0.038879696, -0.0060329405, -0.012260817, 0.02492495, -0.0059018624, -0.025126608, 0.017651813, -0.023715002, 0.035169184, -0.006291735, -0.018324006, -0.0052599176, 0.022088293, -0.009417437, -0.034228116, 0.004755772, -0.0046011675], "content_hash": "c7aa929eff6fa5ed76ea4f783e051e52"}, {"content": "# Key Terminology\n\nAn extended glossary of terms you may encounter while using the Transformer Explanation Dashboard.\n\n## Core Concepts\n\n**Token**: A small piece of text that the model processes. Can be a word, part of a word, or a punctuation mark. The model's fundamental unit of input and output.\n\n**Embedding**: A vector (list of numbers) that represents a token's meaning. Similar tokens have similar embeddings.\n\n**Attention**: The mechanism that lets each token look at all other tokens to gather relevant context. Uses Queries, Keys, and Values.\n\n**Attention Head**: One instance of the attention mechanism. Each layer has multiple heads that look for different patterns simultaneously.\n\n**Layer**: One complete processing step in the Transformer, containing both attention and MLP components. GPT-2 has 12 layers; larger models have more.\n\n**MLP / Feed-Forward Network (FFN)**: The component in each layer that processes tokens individually, storing and retrieving factual knowledge. Uses an expand-then-compress pattern.\n\n## Architecture Terms\n\n**Residual Stream**: The \"conveyor belt\" of information running through all layers. Each layer reads from it and adds back its contribution. This preserves information from earlier layers.\n\n**Layer Normalization (LayerNorm)**: A technique applied before or after each sublayer that stabilizes the numbers, keeping them in a reasonable range. This helps training and makes the model more robust.\n\n**Parameters / Weights**: The learnable numbers in the model. These are adjusted during training to improve predictions. GPT-2 has ~124 million parameters.\n\n**Hidden Dimension**: The size of the internal vector representations. For GPT-2, this is 768 -- meaning each token is represented by 768 numbers at each layer.\n\n**Vocabulary**: The complete set of tokens the model knows. GPT-2 has a vocabulary of about 50,257 tokens.\n\n## Training and Inference", "source_file": "key_terminology.md", "chunk_index": 0, "embedding": [0.0044527943, 0.03728695, 0.034718715, -0.01653895, 0.05407559, -0.009054213, 0.018179767, 0.047916584, -0.0072528827, 0.060638852, 0.010308605, -0.014755455, -0.059449855, -0.0251354, -0.0028075194, -0.030961486, -0.0058974256, 0.039664943, 0.015801772, 0.0663936, -0.006783229, -0.0250165, 0.003679946, 0.06415828, 0.011539217, -0.005855811, 0.011604612, 0.043445956, 0.016027682, -0.0047976035, 0.023292454, -0.017513927, -0.056168225, -0.009357408, -0.00035725647, -0.030200528, -0.018584026, 0.056215785, -0.012912509, 0.03381508, -0.0064265295, -0.019440103, -0.0425661, 0.07657141, -0.013887486, 0.0136140175, -0.021806207, -0.015445073, -0.056929182, 0.059640095, -0.023470804, 0.006563264, -0.028131671, 0.011604612, 0.016051462, 0.015730431, -0.036858913, -0.010486955, -0.020070272, -0.03341082, 0.015373733, -0.011390593, 0.03234072, 0.014541435, -9.902117e-05, -0.02853593, -0.038642406, 0.056168225, -0.03819059, 0.017192898, 0.04156734, 0.020462641, -0.055882864, 0.017359357, 0.01598012, 0.0004607364, -0.027798753, 0.04261366, -0.02863105, -0.033529717, 0.0012387864, -0.04154356, 0.0241842, -0.039498486, -0.0013049244, -0.013602127, -0.031698663, -0.01690754, -0.032483403, -0.058926698, -0.014696005, -0.023768052, -0.0247787, 0.01665785, 0.01689565, 0.018988283, -0.01062369, -0.035788815, -0.027251814, 0.0125677, 0.05402803, -0.0056863786, 0.0075322967, -0.025753679, 0.022115346, -0.058260858, -0.014303636, -0.03709671, -0.043255717, -0.034932736, -0.10700974, -0.012098046, 0.004764906, 0.018358115, 0.053029273, -0.011497602, -0.039569825, 0.011069563, -0.015373733, 0.007330167, -0.005579369, -0.00815652, 0.01253203, 0.008715349, -0.025848798, -0.0069318535, -0.016051462, 0.033648618, 0.0013160713, -0.0066940538, 0.0109744435, 0.0028372444, 0.024136642, -0.036430873, -0.0076690316, -0.02033185, -0.05859378, 0.021770537, -0.04147222, -0.038880207, 0.0010864461, -0.052315872, 0.023399463, -0.009755721, -0.038785085, 0.022531496, -0.017204788, 0.07376538, -0.03750097, 0.0047678784, 0.004577639, -0.009280123, -0.017823067, -0.031127945, -0.016943209, -0.043113034, 0.02085501, 0.09240886, 0.056834064, -0.02901153, -0.0032310998, 0.039284464, 0.015147824, 0.04116308, -0.019451993, 0.01242502, -0.019963263, -0.0016304123, -0.027299374, -0.100351356, -0.008435935, -0.024065303, -0.013197868, -0.012650929, 0.0039712503, -0.010124311, -0.050793957, -0.031555984, -0.021544628, -0.030604785, -0.044944093, -0.0069734682, -0.04477763, 0.023744272, -0.034718715, -0.018405676, -0.001471384, 0.01702644, -0.0018236244, 0.04190026, -0.014505765, 0.046418447, 0.06301685, -0.01246069, -0.00834676, -0.03360106, 0.03305412, -0.051459797, 0.026467076, 0.022650395, 0.029891388, 0.00621251, 0.04855864, 0.002279902, 0.020367522, 0.0006480034, -0.027394494, 0.0045092716, -0.04499165, -0.0497952, 0.017704166, -0.035646133, 0.00010756708, -0.03196024, -0.075810455, 0.021199819, -0.06168517, 0.02079556, 0.031936463, 0.0027406383, 0.0013316768, 0.023898842, 0.013102748, 0.01632493, -0.04187648, 0.014577105, 0.01665785, -0.040925283, 0.0071934327, -0.0021550574, 0.01613469, 0.008013841, 0.0033589168, -0.030866366, -0.03360106, -0.060258374, 0.017834958, -0.017502038, 0.005609094, -0.026776215, -0.01640816, 0.038452167, -0.023423243, 0.04532457, 0.008055456, 0.008388375, -0.007455012, -0.044397153, 0.04984276, 0.04085394, 0.06919964, -0.01262715, -0.035598572, 0.030366987, -0.029582249, -0.0064205844, -0.005918233, 0.031223064, -0.012781719, 0.010772314, 0.011884026, 0.0059866006, -0.035812594, -0.03645465, -0.017644718, -0.019380653, -0.030723685, 0.0020421026, -0.027180474, -0.055787746, 0.02427932, 0.007823601, 0.0017626883, 0.020343741, -0.035788815, -0.008186245, -0.0055407267, 0.015254833, -3.0026822e-05, 0.0015456963, 0.013935046, -0.019820582, 0.06739236, 0.025967697, -0.031651102, 0.0085310545, 0.008007896, 0.015445073, 0.021330608, -0.042994138, 0.02014161, 0.013839927, 0.04879644, 0.0011191436, 0.0064503094, -0.00086722476, 0.025610998, 0.010546405, 0.031009044, -0.02095013, -0.0031954297, -0.026871335, -0.031009044, 0.011129013, 0.0053772395, 0.027346933, -0.0071577625, 0.015124043, -0.015135934, -0.022388816, 0.016241701, -0.059735216, -0.0071815425, 0.011913751, -0.011134958, 0.039617386, 0.029629808, 0.00074572413, 0.01262715, -0.004226885, -0.024326881, -0.05897426, -0.006289795, -0.014696005, 0.048939124, 0.012983848, -0.044516053, 0.0027361796, -0.0019559003, -0.018346226, 0.07814089, -0.023351904, -0.01267471, -0.0028090058, -0.08475172, -0.0123774605, -0.034385797, 0.03215048, -0.03766743, -0.0076155267, -0.031199284, -0.039213125, -0.04606175, -0.011432207, -0.05902182, 0.03609795, 0.014184736, 0.010843654, 0.060686413, 0.0060519953, -0.026514636, -0.0016274399, -0.0051661925, -0.03721561, -0.008840194, 0.004286335, 0.031555984, 0.0012618232, -0.00032660266, 0.005029458, -0.043850213, 0.006206565, 0.023149773, 0.019142853, -0.039831404, -0.019059625, 0.019190414, -0.00066249433, 0.01032644, -0.023910733, 0.019582782, 0.044872753, 0.0047143735, -0.003528349, -0.04575261, 0.004987843, -0.012044541, 0.0059984904, -0.01653895, 0.013459448, 0.05031836, 0.009720052, -0.008025731, 0.027489614, 0.021485178, 0.025753679, -0.011491657, 0.005362377, -0.012127771, 0.05754746, 0.009190948, -0.017347468, -0.02042697, 0.012948179, 0.02103336, 0.034314457, 0.0102551, 0.038689967, 0.059497416, 0.03317302, 0.007948446, -0.007151818, -0.018084647, -0.0061352253, 0.055407267, 0.010439395, 0.043184377, 0.012829279, 0.01678864, 0.034647375, 0.055549946, -0.007092368, 0.004348757, -0.0101897055, 0.003917746, 0.0051156604, 0.035384554, 0.02896397, 0.011747291, 0.04601419, 0.017775508, -0.006147115, -0.059973016, 0.0076630865, -0.034885176, 0.046870265, 0.04201916, -0.03253096, -0.03410044, -0.044753853, 0.0018592943, 0.02884507, 0.04615687, 0.00401881, 0.006194675, -0.040568583, 0.026229277, 0.051745154, -0.02033185, 0.002129791, 0.048629984, -0.02844081, 0.014077726, -0.029962728, -0.055169467, -0.0072885524, -0.03678757, -0.06268393, -0.003540239, -0.02872617, 0.010344275, 0.018631585, 0.004595474, 0.0149575835, -0.013233538, -0.031698663, 0.046347108, 0.06420585, 0.05735722, -0.007728481, -6.929624e-05, -0.025087839, -0.028250571, -0.026039038, -0.036740012, -0.014053946, -0.008911533, 0.013090858, 0.014921914, -0.0075620217, 0.0044022617, -0.0070864228, -0.013114639, -0.044135574, 0.0016437885, 0.0057458286, 0.02970115, -0.006007408, 0.019773023, -0.026728656, 0.0034153943, -0.014149066, -0.009292013, -0.006699999, 0.007098313, 0.10177816, -0.010617744, -0.026966454, 0.031627323, -0.025396978, -0.024564682, -0.027537173, 0.0036888637, -0.011604612, 0.017597158, -0.0048956955, -0.061970532, -0.06677408, 0.028345691, -0.0060876654, 0.018821824, 0.0033589168, -0.00010886755, 0.07990061, -0.0018013307, 0.020319961, 0.043017916, 0.047536105, -0.027204255, 0.02494516, -0.034647375, 0.01250825, 0.00521078, 0.018358115, -0.013447558, 0.047726344, 0.04211428, -0.0025325639, 0.0164795, 0.0059271506, 0.016182251, -0.009262288, -0.02106903, -0.0015092832, 0.009999466, 0.014303636, -0.034266897, -0.048392184, -0.04572883, -0.011010113, 0.0027792808, -0.03657355, -0.0028387306, -0.007865216, -0.026086597, 0.01684809, 0.0062422347, -0.009969741, -0.04508677, -0.015813662, 0.012650929, 0.027418274, 0.04168624, -0.0035788813, 0.02896397, 0.013055189, -0.00067847146, -0.02951091, -0.014660334, 0.01661029, 0.0010871893, -0.021639748, 0.027180474, -0.042661216, -0.030676126, -0.0057963612, 0.00630763, -0.0012075752, 0.0044349595, -0.016943209, 0.0065573193, -0.0013926129, -0.005736911, -0.023518363, -0.042637438, 0.03640709, -0.004524134, 0.0016958072, 0.019642232, -0.040235665, -0.0040931227, -0.025539659, -0.0065335394, 0.03276876, 0.017252348, -0.019737352, 0.019035844, 0.024540901, 0.0007646738, 0.013578347, -0.034029096, -0.0038731582, -0.034219336, -0.020105941, -0.024053412, 0.020308072, -0.0011444098, 0.0073717823, -0.04511055, -0.00502054, 0.047892805, 0.005546672, 0.0015561, 0.019297423, -0.034552258, 0.0058290586, 0.04128198, -0.01674108, -0.00810896, 0.030795027, -2.278137e-05, -0.004776796, -0.036549773, 0.004348757, -0.020070272, 0.003516459, -0.013982606, 0.021556517, 0.0065038144, 0.019071514, -0.009155278, -0.02078367, -0.017121559, 0.0022873331, -0.05778526, 0.03448092, -0.03231694, 0.019309314, 0.023934511, -0.01255581, 0.021604078, -0.014755455, -0.018893164, -0.034813836, 0.0060638855, 0.0334346, -0.035503455, -0.028107893, -0.009416858, 0.007829546, -0.05483655, 0.019285534, 0.064729005, 0.01678864, -0.040069204, 0.02877373, 0.0038166807, -0.0052940096, 0.022472046, -0.022483936, -0.009743831, -0.029320668, -0.019428214, 0.0037304785, 0.0077641513, 0.021948887, -0.02511162, 0.042399637, -0.0040693427, -0.012650929, -0.014743565, -0.017395029, 0.04194782, 0.0021030386, 0.0050889077, -0.0026261974, -0.009416858, 0.000588182, -0.008400265, 0.02506406, -0.040069204, -0.0143987555, -0.003917746, -0.03795279, -0.018453235, 0.039379586, 0.009250398, 0.014684115, 0.0052969824, -0.019701682, -0.072148345, 0.03314924, -0.020748, 0.038285706, 0.013007629, -0.02439822, -0.0009415371, -0.04163868, 0.0014446315, 0.03664489, -0.014684115, 0.0337913, -0.03735829, -0.009280123, -0.008269475, -0.0045687216, 0.023256784, -0.022519605, 0.04941472, 0.019570893, -0.017573377, -0.010284825, -0.0029323641, -0.05807062, -0.054931667, 0.019214194, 0.015219163, 0.050841518, -0.014660334, -0.09155278, -0.010706919, 0.01231801, -0.056691382, 0.0048719156, -0.03381508, -0.01672919, -0.018584026, 0.040259443, 0.0076571414, -0.064253405, -0.0495574, 0.029320668, 8.8849665e-05, 0.022174796, -0.0034064767, 0.008834248, -0.013292988, 0.014969474, -0.006289795, 0.01250825, -0.030200528, 0.013911267, -0.021211708, 0.02064099, 0.018524576, 0.0028357583, -0.02130683, -0.014826794, -0.018893164, 0.017395029, 0.0034510642, 0.025420759, -0.025610998, -0.039403364, -0.0008040593, 0.0018147068, -0.0132216485, 0.0009214728, -0.03754853, -0.010748534, -0.02427932, -0.008834248, -0.017716058, 0.010391835, 0.022020226, 0.023007095, 0.00084716047, -0.015528303, -0.011687842, -0.04917692, 0.004021783, 0.0077879312, -0.013816147, -0.021889437, 0.015278613, 0.00826353, 0.027299374, -0.006705944, 0.027465833, 0.0030378876, -0.0036264414, -0.005368322, -0.027941432, 0.020212952, 0.002351242, -0.051126875, -0.0026247113, 0.012746049, 0.02427932, 0.027656073, 0.002762932, -0.002496894, 0.0023215169, 0.0037364236, 0.02002271, -0.052363433, 0.011455988, -0.007728481, 0.0064205844, -0.008644009, -0.024243651, -0.004764906, 0.0068307887, -0.031341966, -0.019154744, -0.0040842053, -0.019190414, -0.009226617, -0.028464591, -0.01036211, -0.027370714, 0.013078968, 0.023304343, -0.03769121, -0.006361135, -0.014220405, 0.022258027, -0.07381294, -0.009077993, 0.023137884, 0.0047797686, 0.021080919, -0.03386264, 0.004595474, -9.1218375e-05, 0.03262608, -0.07485926, 0.00083081174, -0.005017568, 0.01663407, 0.007026973, 0.023102215, -0.024148531, 0.01662218, 0.026823776, 0.030200528, -0.0072826073, -0.022555275, 0.0023423242, 0.026799995, 0.008542945, -0.02437444, -0.009054213, 0.010980388, -0.014410646, 0.0054069646, 0.0009876108, -0.010825819, 0.03274498, -0.0076095816, 0.025563437, -0.01218722, 0.011325197, -4.6398756e-05, -0.014446315, 0.010790149, -0.026039038, 0.0328401, -0.035812594, 0.023233004, -0.010171871, 0.042899016, 0.0007691325, 0.006991303, -0.0031924574, 0.011479767, 0.034362018, -0.006729724, 0.012109936, -0.011194408, -0.0021193875, 0.0115927225, 0.02886885, 0.0029085842, -0.009648711, 0.021211708, 0.02929689, -0.0021312775, 0.018548355, 0.00803762, 0.0095417015, -0.0073004426, -0.0046222266, 0.055454828, 0.007728481, -0.0137210265, -0.000118713935, 7.914262e-05, -0.029891388, -0.007900886, -0.012757939, -0.0044230693, 0.019226084, -0.011670006, 0.007508517, -0.0055407267, -0.030557226, 0.034623597, 0.003950443, -0.031484645, 0.030105408, 0.025373198, 0.015433183, -0.02874995, -0.02473114, 0.0017582295, 0.025420759, 0.05440851, 0.011717566, -0.01690754, 0.001507797, 0.0045449417, 0.014850575, 0.02437444, 0.03362484, 0.061066892, -0.024422001, 0.06082909, 0.025825018, -0.043231934, -0.040806383, -0.02910665, 0.01218722, -0.018964505, -0.01267471, -0.02905909, 0.008804523, 0.0057874434, 0.009559537, 0.036193073, 0.016003901, -0.01603957, 0.007847381, 0.038903985, 0.047250748, -0.008043566, -0.016087132, -0.0011213729, 0.00512755, -0.014303636, 0.01690754, -0.02903531, -0.017763617, 0.023934511, 0.028274352, 0.014826794, -0.015801772, -0.019190414, 0.026371956, -0.026799995, -0.01619414, -0.008471604, 0.034742497, -0.00096606015, 0.021615967, -0.022269916, 0.014755455, -0.031389523, -0.015623422, -0.0057696085, -0.009161223, -0.026966454, 0.017537707, -0.0018176794, -0.021936998, 0.012014816, -0.0039801677, 0.013709137, 0.015992012, -0.030866366, 0.035313215, 0.011527327, 0.019202303, 0.0026470048, -0.047631226, -0.032364503, 0.039926525, 0.0015211732, 0.042756338, 0.030176748, -0.005778526, 0.024422001, -0.0042803898, -0.013399998, 0.009161223, -0.007021028, -0.003021539, 0.0012276395, 0.011895916, 0.018655365, 0.03664489, 0.035360776, 0.02124738, -0.036977813, 0.034861397, 0.03381508, 0.0032102922, 0.008965039, 0.03269742, 0.025087839, -0.012757939, -0.02520674, -0.00828731, -4.205613e-05, -0.014220405, -0.013923156, 0.026966454, 0.01038589, 0.023209224, 0.0059806556, 0.0065335394, -0.0136140175, -0.031936463, 0.033719957, 0.00033682058, -0.0057963612, -0.008566724, -0.01224667, 0.015028924, -0.008126795, -0.028274352, -0.008602394, 0.022460155, -0.01255581, 0.012888729, 0.028226793, 0.008632119, 0.0055110017, -0.0055585615, -0.0061114454, 0.017549599, -0.02936823, -0.0064740893, 0.01674108, -0.013292988, -0.0028550792, 0.019808693, -0.05069884, -0.022115346, 0.00077582063, -0.019321203, 0.01603957, 0.026776215, -0.007930611, 0.011854301, -0.0012432451, -0.0038434332, -0.011996981, 0.0052405046, -0.0045806114, 0.010784204, 0.028250571, 0.030057847, 0.00821597, 0.009339573, 0.06458633, 0.006991303, 0.008941258, 0.021485178, -0.0029026393, -0.005368322, -0.004286335, 0.0040633976, 0.009684382, 0.052078076, -0.00078845373, -0.035955273, 0.007906831, 0.014470095, 0.015801772, -0.03766743, 0.018762374, 0.019475773, 0.050033, 0.023387574, -0.0011979146, -0.00847755, 0.03735829, 0.035360776, 0.048011705, -0.020224841, 0.015706653, -0.0064384197, -0.0075620217, 0.0019529278, 0.027798753, 0.018262995, -0.03659733, 0.05274391, -0.026538417, -0.003552129, -0.0036829186, -0.027465833, 0.00032771734, -0.014612774, 0.0012216945, -0.027703634, -0.02877373, -0.029558469, 0.017430698, -0.0037304785, 0.0039118007, 0.011693787, -0.018488906, 0.010736644, -0.010183761, -0.03417178, -0.015754212, -0.005644764, 0.016669739, -0.0064859795, 0.011360868, 0.009642767, -0.009101773, -0.005133495, -0.008061401, -0.028702391, -0.0019900838, -0.0147911245, -0.005445607, -0.01622981, 0.016669739, 0.014065836, 0.017181009, -0.047987923, -0.000648375, 0.009690327, 0.0007832519, -0.00839432, -0.014327415, 0.039664943, 0.029938947, -0.015147824, 0.0056299013, -0.042899016, 0.03398154, -0.018429456, 0.020189172, 0.0251354, -0.010837709, -0.0025563438, 0.017763617, -0.03405288, 0.03800035, -0.009714107, 0.004155545, 0.023958292, -0.0034243118, 0.017620938, 0.007799821, -0.006539484, -0.04563371, -0.015932562, -0.014386865, -0.021199819, 0.017288018, -0.024172312, -0.018227326, 0.004964063, 0.0010619231, -0.023328124, 0.030913925, -0.01651517, 0.0011250885, -0.00307653, 0.008911533, -0.006043078, 0.052839033, -0.005942013, -0.0069496883, 0.0002641803, -0.030890146, 0.02473114, -0.030723685, -0.0033381095, -0.033529717, -0.02070044, 0.01705022, 0.028464591, -0.017989527, -0.020224841, 0.007954391, 0.019202303, -0.027323155, 0.009862731, 0.019249864, 0.020165391, 0.014648445, 0.00021123278, -0.014410646, -0.004940283, 0.000491576, 0.009714107, 0.006634604, 0.0008367567, 0.035717472, -0.0048273285, -0.0125677, -0.03764365, 0.0057190764, 0.036169294, -0.0071577625, 0.021378169, -0.015885001, 0.004747071, -0.03750097, 0.0059509305, 0.027846312, -0.026419517, -0.0031032825, -0.0004354702, -0.030890146, 0.0076274164, 0.012912509, -0.011699731, -0.013269208, 0.009696271, -0.0050651277, -0.0072944975, -0.014410646, -0.005888508, 0.045134332, -0.05379023, 0.014196626, 0.034742497, -0.0050681, 0.012841169, -0.026847554, 0.03348216, -0.0020970937, 0.009838951, -0.01051668, -0.036502212, -0.0143987555, 0.02083123, -0.0063670794, -0.008715349, -0.002991814, -0.013233538, -2.027333e-05, -0.01690754, 0.016396271, 0.025848798, -0.026633536, 0.010379945, -0.0018890192, 0.02070044, -0.022341257, -0.009838951, -0.017834958, 0.018643474, 0.0021134424, -0.02470736, 0.039831404, -0.03250718, -0.014767344, 0.025920138, -0.0050621554, 0.009363352, 0.0016854034, 0.044088013, -0.003269742, -0.011224133, -0.018203545, -0.019428214, 0.023910733, 0.0032489346, -0.005442634, 0.0040574525, -0.022590946, 0.0007490682, -0.0045627765, -0.010171871, -0.017775508, -0.0059063435, -0.0017537707, 0.014600885, -0.017371248, -0.0034718716, 0.0067891735, -0.0036056337, 0.030295648, 0.03286388, 0.016241701, 0.060068134, -0.019570893, -0.02014161, 0.026443297, -0.02886885, 0.01674108, -0.010766369, 0.0032132647, -0.014529545, 0.02078367, -0.0023423242, -0.027560953, -0.026015257, 0.0021996447, 0.01628926, -0.0018414593, -0.028036552, 0.0024998665, 0.020153502, 0.004342812, 0.021520847, -0.021592189, 0.01631304, -0.00010366569, 0.010540459, 0.00035075416, 0.03329192, 0.060305934, 0.02029618, -0.0005090394, -0.0035937438, -0.029891388, -0.01235368, -0.010813929, -0.02970115, -0.011378703, 0.018179767, 0.018298665, -0.06306441, 0.0034837616, 0.003153815, 0.0030913926, -0.014303636, 0.018536465, 0.028345691, -0.029867608, 0.0047738235, 0.013578347, -0.010463174, -0.017216679, 0.014374975, 0.032959, 0.0019365791, -0.008756964, -0.0022754432, -0.009416858, 0.02544454, -0.003668056, -0.0095417015, 0.007859271, -0.01238935, -0.026443297, 0.0013257317, 0.013150308, 0.012829279, -0.018560246, -0.0097973365, -0.020058382, 0.026942676, -0.01675297, -0.0049046134, 0.007621472, -0.015266723, 0.00025990733, -0.048273284, -0.010998224, 0.0018236244, 0.021378169, 0.04168624, 0.0036710287, 0.038903985, 0.03381508, -0.0035788813, 0.0016943209, 0.0009980145, -0.0065038144, -0.010308605, -0.038737528, 0.038737528, -0.017644718, 0.003798846, 0.0006182785, -0.0020004876, -0.015385623, -0.027109135, -0.009274177, -0.020094052, 0.013792367, 0.013625907, 0.00523456, -0.019570893, -0.0035967163, -0.010885268, -0.021294938, -0.0076274164, 0.042946577, -0.036502212, -0.0008003437, -0.009743831, -0.025872577, -0.02482626, 0.009482252, 0.0046341163, 0.009613042, -0.0409015, 0.013257318, 0.00815652, -0.015944451, -0.014089616, -0.012888729, 0.0017062109, -0.005192945, 0.01619414, -0.008031676, 0.026276836, -0.0035075415, 0.035574794, -0.010813929, 0.022900084, 0.01656273, 0.007151818, -0.060686413, 0.0064086947, 0.033957757, 0.009892456, 0.00831109, -0.027965212, -0.02045075, -0.025729898, -0.0025236465, 0.00845377, 0.005029458, -0.014731674, -0.025087839, -0.0051126876, -0.004565749, -0.014065836, -0.0009868676, 0.02451712, -0.0038345158, -0.019309314, -0.02951091, 0.027156694, -0.0005993289, -0.02898775, -0.009214728, -0.0140063865, 0.017573377, 0.040877722, 0.009464418, 0.01266282, 0.01724046, -0.0022278833, -0.01661029, 0.014279855, 0.0016809447, -0.00399503, 0.02492138, 0.010118365, 0.008661844, 0.01663407, 0.009624932, -0.012008871, -0.015968231, 0.018465126, -0.011414372, -0.03238828, -0.0013651174, -0.0027376658, -0.015956342, -0.03652599, -0.0055823415, 0.047250748, -0.011027948, 0.005620984, 0.02128305, 0.0058974256, 0.015064593, 0.051697597, -0.029439569, 0.04206672, 0.0036502213, -0.007080478, -0.015670983, -0.022757405, -0.01653895, -0.009083938, -0.01643194, -0.02105714, -0.009137443, -0.008709404, 0.011390593, 0.042589877, 0.010605854, 0.013851817, 0.008328925, 0.0028075194, 0.074336104, 0.003727506, 0.009000708, 0.012056431, 0.037596088, -0.012757939, 0.019475773, 0.02439822, 0.014018276, 0.019582782, 0.008305145, -0.0014327415, 0.030676126, 0.035289437, 0.02033185, -0.015338063, -0.03312546, 0.0032073196, 0.0009222159, -0.035027854, 0.005499112, 0.010689084, -0.022198576, -0.01219911, -0.015647203, -0.0011815659, 0.010790149, 0.046727587, -0.02097391, -0.00823975, 0.0067118886, 0.0137210265, 0.012757939, -0.00808518, 0.003275687, -0.055597506, -0.013887486, -0.002262067, 0.015421293, -0.01638438, 0.0064146393, 0.03700159, 0.009280123, -0.067107, -0.02072422, 0.0136140175, 0.00067847146, 0.018500796, 0.025848798, 0.011093344, -0.009012599, -0.010463174, -0.027822534, -0.022781186, 0.0032429898, 0.027109135, 0.027679853, -0.022959534, -0.0043398393, -0.009559537, 0.020498311, -0.0137210265, 0.007437177, 0.022876306, 0.025278078], "content_hash": "d39c81a382c4f7945e6cc285198af819"}, {"content": "~124 million parameters. **Hidden Dimension**: The size of the internal vector representations. For GPT-2, this is 768 -- meaning each token is represented by 768 numbers at each layer. **Vocabulary**: The complete set of tokens the model knows. GPT-2 has a vocabulary of about 50,257 tokens. ## Training and Inference\n\n**Training**: The process of adjusting the model's parameters by showing it billions of text examples. The model learns to predict the next token and its parameters are updated to reduce prediction errors.\n\n**Inference**: Using the trained model to make predictions on new text. This is what happens when you click \"Analyze\" in the dashboard -- no learning occurs, the model just processes your input.\n\n**Forward Pass**: One complete trip of data through the model, from input tokens to output predictions. The dashboard visualizes this forward pass.\n\n**Gradient**: A measure of how much each parameter contributed to the model's prediction error. Used during training to update parameters, and in attribution experiments to measure token importance.\n\n**Loss**: A number measuring how wrong the model's predictions are. During training, the goal is to minimize this. Lower loss means better predictions.\n\n**Fine-tuning**: Taking a pre-trained model and training it further on a specific dataset to specialize its behavior.\n\n## Prediction Terms\n\n**Logits**: The raw, unnormalized scores the model assigns to every possible next token before converting to probabilities.\n\n**Softmax**: The function that converts logits into probabilities (positive numbers that sum to 1.0).\n\n**Probability Distribution**: The complete set of probabilities over all possible next tokens. The dashboard shows the top 5.\n\n**Temperature**: A setting that controls prediction confidence. Low temperature = more focused; high temperature = more spread out.", "source_file": "key_terminology.md", "chunk_index": 1, "embedding": [-0.01888086, 0.0065460494, 0.053035837, -0.013798045, 0.019882021, -0.012861062, 0.001687854, 0.040611178, 0.014709358, 0.06638464, 0.008156891, -0.009241481, -0.040226117, -0.031061646, 0.009177305, -0.019176073, -0.015582165, 0.03747934, -0.020857511, 0.064741716, -0.03052256, -0.030240182, -0.00554168, 0.038942575, 0.01478637, -0.008278827, 0.0038281553, 0.03139537, 0.05182931, -0.00076290354, 0.03614446, -0.01821342, -0.044513136, -0.052394066, 0.0147607, -0.042844534, -0.0064979168, 0.0590428, -0.005859356, 0.02690298, -0.043306608, -0.01673735, -0.08476492, 0.05421669, -0.036580864, 0.01645497, 0.02297535, -0.021653304, -0.045539968, 0.035836414, -0.032601893, 0.044179417, -0.053959984, 0.018457292, 0.014170271, 0.009908922, -0.017160919, -0.012039597, -0.008991191, -0.04692619, -0.00017768994, -0.027955482, 0.038993917, 0.044436123, 0.020703485, -0.025811972, -0.043820024, 0.070132576, -0.03968703, 0.00893985, 0.018316103, 0.00217239, -0.014349967, -0.03018884, 0.0013204409, 0.0065685115, -0.0021001908, 0.016531983, -0.019342935, -0.036375497, 0.026197033, -0.021435102, -0.015517987, -0.03968703, -0.021871507, -0.0453346, -0.00991534, -0.048235398, -0.03278159, -0.04797869, -0.03709428, -0.02345026, -0.022359252, 0.022423428, 0.008400763, -0.022064038, 0.03411647, -0.013464325, -0.020292753, -0.014426979, 0.01094217, 0.014619511, 0.016647503, -0.013066428, 0.015479482, -0.031035975, -0.06535781, -0.010621286, -0.019227415, -0.029778108, -0.0879481, 0.025439745, -0.049775645, 0.006597391, 0.0112181315, -0.043537647, -0.035425678, 0.034655556, 0.016852869, -0.02378398, -0.01835461, -0.02280849, 0.029316032, 0.010197719, -0.023013856, 0.0039789714, -0.009119545, -0.02428456, 0.0050667706, 0.01626244, 0.0051983334, 0.03486092, 0.011539017, -0.045026552, -0.0044153747, -0.026517918, 0.008952686, 0.052034676, -0.035271656, -0.026197033, -0.0081697265, 0.0045565637, 0.017802687, 0.03350037, -0.06325281, -0.00818898, -0.02211538, 0.06499842, -0.049595952, 0.030086156, -0.0004001032, 0.016172593, 0.008914179, -0.034835253, 0.007078718, -0.04043148, 0.017507473, 0.0760882, 0.015851708, -0.040508494, -0.00013176331, 0.027288042, 0.021820165, 0.017828358, -0.016429301, 0.032396525, -0.04161234, -0.0034463024, -0.05704048, -0.08430285, 0.018559976, -0.0073418436, -0.025375567, -0.0137852095, -0.0035489856, -0.025914654, -0.079682104, -0.021242572, 0.006013381, -0.0717755, -0.063458174, -0.02152495, -0.034373175, 0.022872666, -0.031600732, -0.021383762, -0.005057144, 0.009536696, 0.019676654, 0.020421106, 0.021576293, 0.026055845, 0.08959103, 0.007316173, -0.03611879, -0.028289203, 0.052984495, -0.0018033724, -0.0112694735, 0.048466437, 0.018675493, 0.024181876, 0.030368535, -0.0538573, 0.0005896258, 0.005852938, 0.008503446, -0.0033532458, -0.050212048, -0.013079263, 0.0011287122, -0.0056732427, 0.007836007, -0.007078718, -0.06874635, 0.021229737, -0.0638689, 0.029932132, 0.01690421, 0.03588775, -0.005333105, 0.02656926, -0.008914179, 0.0105314385, -0.008548371, 0.018059395, 0.029136337, -0.0015787531, 0.051957663, 0.0010781728, 0.0026328596, -0.012828973, -0.019753667, -0.027647432, 0.007945107, -0.03116433, 0.038146783, -0.013515667, 0.006395234, -0.0430499, 0.014272955, 0.033988114, -0.007251996, 0.021281078, 0.041329958, 0.010383831, -0.01854714, -0.037941415, 0.0505201, 0.016044239, 0.051418576, -0.011384992, -0.022449099, 0.012925238, 0.0036580865, -0.019843515, -0.013348807, 0.031883113, 0.00089045527, -0.016929882, 0.029701095, -0.008355839, -0.039507333, -0.023103705, -0.016660338, -0.0030179212, -0.016852869, -0.023899497, -0.03704294, -0.0879481, 0.021768823, 0.004832525, -0.015312621, 0.014195942, -0.04027746, -0.021062877, -0.0031222089, 0.015543658, -0.024990506, 0.021422267, 0.0033211575, -0.007483033, 0.06279074, 0.00156672, 0.0170454, -0.0052657193, 0.022859832, -0.0088564195, 0.0003451517, -0.029110666, 0.01971516, -0.004951252, 0.02823786, 0.010967841, 0.0071557304, 0.05775926, 0.029932132, 0.022731477, 0.0019573972, -0.009658632, 0.02526005, -0.058221333, -0.016275275, -0.013772375, 0.028417556, 0.04143264, 0.012289887, 0.042895876, 0.0033500372, -0.018316103, -0.008541953, 0.0066358973, -0.00048734376, 0.0012458351, -0.0076819817, 0.03486092, 0.030394206, -0.0029729975, 0.042844534, -0.024425749, -0.0130215045, -0.023552943, 0.013047175, 0.022628793, 0.03668355, 0.015440975, 0.00015572939, -0.006725745, 0.017648663, -0.006995288, 0.048132718, -0.002369734, -0.030060485, 0.049441926, -0.047850337, -0.0064176954, -0.023219222, 0.044282097, -0.05121321, 0.026748955, -0.04987833, -0.008721648, -0.037864402, -0.010595615, -0.013387313, 0.010691881, 0.010781729, 0.04004642, 0.06602525, 0.010865158, 0.0118278125, -0.005692496, 0.011917661, -0.0029457223, 0.03193445, 0.014401309, 0.019291593, 0.017777016, 0.0029890416, -0.015492317, -0.06525513, 0.003937256, 0.041894715, 0.007842424, -0.0137081975, -0.020318424, -0.01306001, -0.020202905, -0.00061288994, -0.02049812, -0.027775787, 0.022744313, -0.020036045, -0.023180716, -0.035579704, -0.020664979, -0.0015867752, 0.018906532, -0.029264692, 0.0019156821, 0.043666, 0.021165559, 0.009530278, 0.032293845, 0.025272885, 0.03550269, 0.023860991, -0.02311654, -0.0010356556, 0.036375497, -0.0039918064, -0.03049689, -0.02595316, 0.006462619, -0.012476, 0.016172593, -0.043563318, 0.058632065, 0.041560996, 0.031677745, -0.041201606, -0.026492247, -0.04299856, -0.021563457, 0.0057855523, 0.03314098, 0.0511362, 0.045077894, 0.058324017, 0.013990576, 0.0638689, -0.010082199, 0.0067578335, -0.023155045, 0.040482823, -0.0069182757, 0.03450153, 0.026748955, 0.038198125, 0.035040617, 0.005378029, 0.0064754547, -0.044872526, -0.013964905, -0.02988079, -0.0044538807, 0.02790414, -0.012264216, -0.015607836, -0.03966136, 0.0024162624, 0.029239021, 0.016724514, -0.014773535, 0.0025558472, -0.05188065, 0.038891234, 0.056629743, -0.02918768, 0.0032425406, 0.037582025, -0.016673174, 0.001608435, 0.0020985864, -0.041150264, -0.01873967, -0.027827127, -0.05416535, 0.009478936, -0.05647572, 0.012379735, 0.05380596, -0.041329958, 0.031857442, -0.029136337, -0.038608856, 0.028340543, 0.019638149, 0.07372648, -0.03450153, 0.014542498, 0.007970778, -0.050391745, -0.0351433, -0.0047715567, 0.0038859146, 0.01707107, 0.005021847, -0.001926913, -0.005589813, -0.007611387, -0.016236769, 0.010518603, -0.04733692, 0.017802687, 0.004450672, 0.018033724, 0.03190878, -0.0049865493, -0.014439815, 0.029983474, 0.0130985165, -0.022333581, 0.01776418, -0.0017873282, 0.06925977, -0.007585716, -0.009324912, 0.036272816, -0.026979992, -0.00902328, -0.048671804, 0.020767663, -0.013515667, 0.0030612408, 0.020254247, -0.022911172, -0.04618174, 0.007829589, 0.016057074, 0.038172454, -0.00952386, -0.017995218, 0.06427964, -0.013271795, 0.056629743, 0.027467737, 0.018264761, -0.02721103, 0.028956642, -0.07562613, -0.0032842557, -0.023758309, -0.0019750458, -0.0136311855, 0.032037135, 0.026389563, -0.0066679856, 0.007861677, 0.002286304, 0.0098190745, 0.020613639, -0.031061646, 0.007354679, 0.018636988, -0.03116433, -0.027878469, -0.05146992, -0.019882021, -0.027288042, 0.032319516, -0.036965925, 0.011282309, 0.017995218, -0.03768471, 0.02278282, -0.0015554889, -0.036401168, -0.024746634, 0.031626403, 0.009729226, 0.010987095, 0.023193551, 0.035554033, 0.006241209, 0.020575132, 0.022949679, -0.020870347, -0.028648593, 0.036632206, 0.039147943, -0.013348807, 0.026723284, -0.036888912, -0.030394206, -0.03193445, 0.005368402, -0.007585716, 0.004505222, -0.017109577, 0.014195942, 0.0069246935, 0.018264761, -0.020934522, -0.025349896, 0.027057005, 0.011404245, -0.02309087, 0.018393116, -0.028674264, -0.03509196, 0.012033179, 0.0019237043, 0.0007107598, 0.007887348, -0.035194643, 0.013964905, -0.0008872464, -0.018444456, 0.019920526, -0.015672013, -0.0019590016, 0.01155827, 0.013900729, -0.041869044, 0.019484123, 0.019381441, 0.0093890885, -0.036401168, 0.006337474, 0.041766364, -0.032447867, -0.0071172244, 0.021833, -0.04420509, 0.005759882, 0.014041917, -0.015890215, -0.018251926, 0.027288042, 0.02069065, -0.014799206, -0.029367374, -0.024900658, -0.009511025, 0.016852869, -0.013567008, 0.04556564, -0.017481802, -0.020819005, -0.022769984, -0.052984495, -0.033628725, -0.01673735, -0.035271656, -0.006719327, -0.025362732, -0.018868025, 0.04530893, -0.015890215, 0.03314098, -0.018251926, 0.0037864402, -0.025401238, 0.019959033, 0.018572811, -0.023655625, -0.017468967, 0.0020456405, -0.0060807667, -0.027416395, 0.018405952, 0.056681085, -0.004607905, -0.009414759, -0.0063823983, 0.012739126, -0.007502286, 0.012013926, -0.032319516, 0.025503922, -0.0015121695, -0.0025751004, -0.02013873, 0.018277597, 0.0023536899, -0.034912262, 0.029393045, -0.025722124, -0.017892536, -0.020228576, -0.020575132, 0.012604354, 0.01673735, 0.00389875, -0.016634667, -0.01690421, -0.011179626, -0.0076563107, -0.0055545154, -0.02954707, -0.0069631995, -0.013964905, -0.015800366, -0.006642315, 0.04166368, 0.017802687, 0.008529117, 0.030779269, -0.021062877, -0.055038158, 0.0471829, -0.0013830133, 0.008984773, -0.009010444, -0.022898337, -0.016724514, -0.075574785, -0.0069182757, 0.05254809, -0.0209987, 0.027519079, -0.0340908, -0.0015049496, -0.00829808, -0.01255943, 0.049364913, 0.0047747656, 0.036170132, 0.0031141867, 0.0021691811, -0.0010396667, -0.0018900114, -0.015607836, -0.065152444, -0.0026023756, -0.0015779509, 0.060891096, 0.016891375, -0.04376868, 0.0069631995, 0.025029013, -0.03771038, -0.029444387, -0.044179417, 0.017340614, -0.019869186, 0.021845836, 0.004412166, -0.04045715, -0.026415234, 0.026440905, 0.0010661397, 0.008394346, -0.011962584, 0.0137081975, -0.010383831, 0.0015185872, -0.014465486, -0.024810812, -0.03152372, 0.014478321, -0.029957803, 0.011866319, 0.0077204877, -0.046412773, -0.029136337, -0.024117699, -0.023437424, 0.005053935, 0.018405952, 0.015440975, -0.019779338, -0.023373246, -0.009812657, 0.0075921337, -0.00088483974, 0.0009971494, -0.011462004, -0.023347577, -0.0099217575, 0.017918207, -0.01578753, -0.0008655867, 0.019342935, 0.01183423, -0.00966505, 0.009350582, 0.0073097553, -0.0690544, -0.003520106, 0.010345325, -0.021447938, -0.022924008, 0.037350986, 0.0025414075, 0.007033794, -0.0030644496, 0.0013084076, 0.026517918, 0.014696523, 0.015312621, -0.007579298, 0.044898197, -0.015107255, -0.053035837, 0.023886662, 0.018200584, -0.024079194, 0.02380965, 0.006401651, -0.009652214, -0.0023873828, 0.026415234, 0.030111827, -0.032730248, 0.020703485, 0.008464941, 0.0060165897, 0.0074894507, -0.015800366, -0.009645796, 0.020677814, -0.015646342, -0.0035361503, -0.03473257, -0.008009284, -0.007194237, -0.040919226, -0.031600732, -0.015363963, 0.013746704, 0.0024579773, -0.02564511, 0.02576063, -0.0010966237, 0.01690421, -0.09220945, -0.009838327, 0.0035361503, -0.019394277, 0.01776418, -0.035554033, 0.04895418, -0.010897247, 0.012257799, -0.04438478, 0.010993512, -0.027493408, 0.019099062, -0.0092093935, -0.0011471631, -0.034296166, -0.0032858602, -0.000916126, 0.010852323, -0.027262371, -0.025965996, 0.011044854, -0.0030917248, -0.0014752678, -0.06356086, -0.008452105, 0.015171432, 0.0021435104, 0.023514437, 0.0024178668, -0.017982382, 0.014362803, 0.008663889, 0.016339453, -0.012777632, 0.021268243, -0.010396667, -0.027955482, 0.019086227, 0.012046015, 0.029932132, -0.037273977, 0.03745367, 0.006616644, 0.054627426, -0.02506752, -0.013682527, -0.009491771, 0.004132996, -0.008862837, -0.0020809378, 0.052933153, 0.0106084505, -0.023193551, 0.011410663, 0.02188434, 0.01083307, -0.023694132, 0.030419877, 0.03830081, -0.022757148, 0.022423428, -0.0012362086, -0.017558815, -0.023270564, 0.006116064, 0.030009145, -0.0036356244, -0.039584346, -0.013541337, -0.015992897, 0.006456202, -0.0059042796, -0.010492932, -0.014144601, 0.03085628, -0.0013412983, 0.0019445617, -0.044256426, 0.0072840843, 0.04240813, 0.0045212666, -0.038891234, 0.02219239, 0.04045715, 0.0721349, -0.016352288, -0.02414337, -0.015376799, 0.009536696, 0.059504874, -0.012103774, -0.013913563, -0.005188707, -0.028366214, 0.0032056388, 0.0071493126, 0.04887717, 0.022230897, -0.0063823983, 0.023527272, 0.0058080144, -0.057964627, -0.012418241, 0.0017536354, 0.009042533, -0.0066936566, 0.005496756, -0.011840648, -0.01921458, -0.014298625, 0.016249605, 0.010114288, 0.013258959, -0.04107325, 0.015569329, 0.0544734, 0.047439605, -0.023848157, -0.05313852, 0.0014592236, 0.0038474086, 0.011314397, 0.03573373, -0.041329958, 0.012090938, 0.0026938277, 0.0386602, 0.0021066086, -0.009883251, -0.04266484, -0.0011632073, -0.008760154, 0.008984773, -0.0019124732, 0.026825966, 0.007264831, -0.010287566, -0.033859763, 0.0099217575, -0.013759539, 0.018328939, -0.013246124, -0.008599712, -0.013515667, -0.007001706, -0.0040848632, -0.046874847, 0.015505153, 0.014581004, 0.031010306, 0.0054037, -0.025555264, 0.022141049, -0.021820165, 0.0216148, 0.017982382, -0.021011535, -0.03445019, 0.041766364, 0.010987095, 0.05226571, 0.021845836, 0.014067588, 0.030933293, -0.03275592, -0.011051272, 0.0066294796, -0.018226255, -0.0002038621, -0.008028537, 0.015235609, 0.00991534, 0.02409203, 0.027390724, 0.043306608, -0.021370927, 0.014670852, 0.013913563, 0.026312552, 0.024964835, 0.017443296, 0.007097971, -0.009549531, 0.010576362, -0.034193482, -0.015607836, -0.0009056973, -0.034681227, 0.041715022, -0.004935208, 0.01971516, 0.016467806, 0.038506173, -0.008497029, -0.033269335, 0.023065198, -0.0038056935, -0.0069567817, 0.013181946, 0.0055320533, -0.014067588, -0.017815523, 0.0015891818, -0.0070209587, 0.032268174, -0.004992967, 0.0051566185, 0.03478391, -0.0029986682, 0.013156276, 0.00091532385, 0.011487675, -0.0098960865, -0.031857442, -0.025157366, 0.0050892327, 0.03283293, -0.026466576, 0.007502286, -0.0426135, -0.001290759, 0.010345325, -0.007001706, 0.010210553, 0.0071364776, 0.017456131, 0.012848226, -0.018393116, 0.015197103, -0.012476, 0.01835461, -0.010409502, 0.0071364776, 0.037505012, 0.029957803, 0.011372156, 0.0046207407, 0.02464395, 0.006616644, 0.027467737, 0.036632206, 0.0023280191, -0.008933432, 0.0012747148, -0.017584486, 0.0049223723, 0.041560996, -0.020292753, -0.017828358, 0.013008669, 0.04523192, 0.02364279, -0.011179626, 0.008073461, 0.0137338685, 0.035554033, 0.039481662, 0.006112855, -0.012828973, -0.010223389, 0.03547702, 0.02013873, -0.015235609, 0.031215671, -0.021653304, -0.0073867673, 0.012347646, 0.041715022, 0.020677814, -0.023732638, 0.021409431, -0.029290363, 0.0070145414, 0.0064465753, -0.027057005, 0.007393185, -0.013438654, 0.012334811, 0.0138108805, -0.035631046, 0.0016557655, -0.014658016, -0.0034110053, -0.012001091, 0.012469582, 0.013682527, -0.0025526383, -0.0021691811, 0.0035682388, 0.00016565676, 0.015338292, -0.010659792, -0.00598771, 0.0021451148, 0.013387313, 0.010018023, 0.018020889, -0.020382602, -0.03157506, 0.024361571, -0.0413813, -0.0071814014, -0.01902205, 0.0063150125, 0.024117699, 0.02606868, -0.036349826, -0.026748955, 0.021717481, 0.0072134896, -0.018303268, -0.0058754003, 0.045462955, 0.031677745, -0.0069631995, 0.004787601, -0.049390584, 0.009324912, -0.0056090658, -0.0044635073, 0.0063791894, 0.009048951, -0.01286748, -0.0074958685, -0.03375708, 0.027313713, -0.01445265, -0.01099993, 0.023578614, 0.032653235, 0.016827198, 0.0034110053, -0.018957872, -0.04420509, -0.01885519, -0.0118278125, -0.022654464, -0.015864544, -0.04559131, -0.0113464855, 0.04004642, -0.020947358, -0.01904772, -0.0010260291, 0.003937256, -0.0037479342, 0.013361642, 0.022449099, -0.024464255, 0.0459507, 0.006680821, -0.022731477, -0.0016750186, -0.026517918, 0.013297465, -0.008343005, -0.037967086, -0.033834092, 0.0021001908, 0.018726835, 0.024348738, -0.030471219, 0.00459507, 0.011705876, -0.0076563107, -0.007393185, -0.020087387, 0.037273977, 0.008111967, 0.011699459, 0.024207547, -0.0004364033, -0.008721648, -0.0009626543, 0.013913563, 0.00921581, -0.032473538, 0.037864402, 0.045154907, -0.022410592, -0.020344095, -0.007258414, 0.0386602, 0.0071557304, 0.019894857, -0.004158667, 0.02102437, -0.030753598, 0.017943876, -0.005400491, -0.0067642513, -0.0033211575, -0.011731547, -0.0072455783, 0.019971868, 0.013746704, -0.03768471, -0.0021755989, -0.009235064, 0.022449099, 0.011121866, -0.028109506, -0.0023905917, 0.026697613, -0.028982313, -0.011847066, 0.020241411, -0.015338292, 0.016172593, -0.049441926, 0.019882021, -0.011442751, 0.018457292, -0.015248445, -0.058940116, -0.00977415, 0.001854714, -0.015928721, -0.011314397, -0.01873967, -0.004027104, -0.019561136, -0.021486444, 0.013913563, 0.020485284, -0.013477161, 0.018945036, -0.009684303, 0.031677745, -0.018418787, -0.016493477, -0.02069065, 0.004473134, 0.010704716, -0.02230791, 0.045668323, -0.0072006546, -0.0064465753, 0.0086318, 0.0027435648, 0.0039500915, 0.002093773, 0.027416395, -0.0007540792, -0.0002661338, 0.0015675222, 0.005670034, -0.0060358425, 0.0051726624, -0.0077525764, -0.009992352, -0.01542814, 0.0077910824, 0.0011744383, -0.012482418, -0.02857158, -0.0048774485, 0.0072455783, -0.0044314186, 0.0020039254, -0.012861062, 0.018110737, -0.007361097, 0.02445142, 0.016493477, -0.0022092918, 0.03840349, -0.017777016, -0.03378275, 0.022962514, -0.00933133, 0.019111898, -0.018726835, 0.0030066904, 0.030445548, -0.014619511, 0.018303268, -0.04045715, -0.0064176954, -0.010890829, 0.012257799, 0.005817641, 0.0042485143, 0.02188434, 0.01099993, 0.0052400483, -0.004251723, -0.042125754, 0.03488659, -0.010281148, -0.007604969, -0.020896016, 0.0037318899, 0.0030644496, -0.014375638, 0.016929882, -0.008105549, -0.0019782546, -0.015569329, -0.028982313, -0.018059395, -0.007739741, -0.0147607, -0.0062508355, -0.032011464, 0.0066487323, 0.007059465, -0.011526181, 0.0014287395, 0.022359252, 0.0031109778, -0.03152372, 0.010114288, -8.5084605e-05, 0.017250765, -0.02364279, -0.0006317419, -0.00061369216, -0.031446707, -0.0066038086, -0.029239021, 3.36929e-05, 0.028006824, 0.004800436, -0.014183107, -0.0019959032, -0.0039821803, -0.03154939, 0.0040559834, -0.0066102264, 0.020613639, -0.0112438025, -0.022590289, -0.0094661005, 0.015299786, -0.019471288, -0.016480641, -0.018778177, -0.02428456, 0.012347646, -0.049775645, 0.009247899, -0.00871523, 0.01723793, 0.026671942, 0.007354679, 0.0050058025, 0.03216549, -0.01838028, -0.03314098, 0.012026762, -0.014439815, -0.007316173, -0.029752437, 0.03350037, -0.029752437, -0.03611879, 0.00083831145, 0.005394073, -0.013721033, -0.014208778, -0.019111898, -0.0043319445, 0.036349826, -0.042716183, -0.0007829589, -0.038172454, 0.008516282, -0.042100083, 0.0093890885, -0.015184267, 0.027698774, 0.010037276, -0.025106024, -0.010512185, -0.013592679, -0.036221474, 0.019586807, 0.0033628724, 0.0024852527, -0.025747795, 0.0111667905, 0.012321975, -0.029803777, 0.00027776588, -0.022359252, 0.026492247, 0.02297535, 0.0036612954, -0.03830081, 0.009690721, 0.0023552943, 0.04302423, -0.0052015423, 0.040149104, -0.003152693, 0.01626244, -0.035631046, 0.0024467465, 0.022256568, -0.000916126, -0.0032409362, -0.04068819, -0.008567624, -0.006116064, -0.011256638, -0.008657471, -0.026312552, -0.0003497644, -0.046720825, -0.011936913, 0.0125658475, -0.031985793, 0.01311777, 0.048646133, 0.0024098447, -0.02823786, -0.042177096, 0.01707107, -0.0014335527, 0.0049833404, -0.017584486, -0.014093259, 0.016236769, 0.040559836, 0.021358091, -0.0025061101, 0.015800366, -0.021910012, -0.0034206316, 0.008959103, 0.018187748, -0.011089778, 0.01640363, 0.008683142, -0.0011295144, 0.014581004, 0.005724584, 0.013207617, -0.022846997, 0.020177234, 0.025811972, -0.026979992, 0.013772375, 0.035990436, -0.0015827642, -0.0017151291, 0.019176073, 0.05283047, 0.003773605, -0.0070145414, 0.027108345, -0.0008130418, 0.013977741, 0.031729087, 0.020921687, 0.012751961, -0.029239021, 0.0017119203, 0.008747319, -0.028135177, -0.017725674, 0.0019750458, -0.012751961, -0.00022201215, -0.012212874, 0.0043640328, -0.008503446, 0.024194712, -0.013489996, 0.0133103, 0.03316665, -0.0039115855, 0.052573763, -0.009350582, 0.008375092, 0.0339111, -0.00046247517, -0.019176073, 0.022230897, 0.04063685, 0.018303268, 0.008227485, 0.016108416, -0.022372087, 0.007251996, 0.024361571, 0.028186519, -0.00047410725, -0.038634527, -0.0021755989, 0.00434478, -0.050597113, -0.025799137, 0.022936843, -0.007868094, -0.033449028, -0.008420017, 0.028751276, -0.011770054, 0.023296235, -0.022436263, -0.014234449, 0.0062893415, 0.011789307, 0.00356503, 0.0023039526, -0.029803777, -0.04792735, -0.023578614, -0.009414759, 0.011737965, 0.0071493126, 0.037556354, 0.020254247, -0.0057021226, -0.06730879, -0.009254317, -0.005714958, 0.0062829237, 0.0027435648, 0.02149928, -0.021268243, 0.004864613, -0.0008992796, -0.040970568, -0.009645796, 0.017802687, 0.034347508, 0.04107325, -0.023963675, 0.005015429, 0.006494708, 0.028648593, -0.0117764715, -0.0069567817, 0.027031334, 0.022423428], "content_hash": "cf4e56cd78a96e88e93eccf8d45837d0"}, {"content": "**Softmax**: The function that converts logits into probabilities (positive numbers that sum to 1.0). **Probability Distribution**: The complete set of probabilities over all possible next tokens. The dashboard shows the top 5. **Temperature**: A setting that controls prediction confidence. Low temperature = more focused; high temperature = more spread out.\n\n**Beam Search**: A generation strategy that explores multiple possible sequences simultaneously instead of just picking the single best token at each step.", "source_file": "key_terminology.md", "chunk_index": 2, "embedding": [0.01611568, -0.0037612289, 0.036948144, -0.014556972, -0.004987864, -0.0042118984, 0.0010749999, 0.01177163, 0.03328857, 0.027894087, 0.00085983047, 0.0110193845, -0.061860356, 0.00528944, 0.035972256, -0.001645114, -0.0065601254, 0.037815597, -0.020602047, 0.015939478, -0.052481003, -0.052291248, 0.0057739946, 0.028707325, -0.004161071, -0.0067871544, 0.025874544, 0.0110193845, 0.03076753, 0.0088168625, 0.009216705, -0.028869973, -0.0150449155, -0.029330809, -0.008186603, -0.029547673, -0.017565956, 0.018989123, 0.017281322, 0.0040661935, -0.05448699, -0.005052245, -0.06299889, 0.027866978, 3.3064294e-05, -0.0045575253, -0.013316783, -0.084143095, 0.0034071312, 0.06045074, -0.022852007, 0.029357916, -0.027243495, 0.06045074, 0.023827894, 0.009840188, -0.011954608, 0.035565637, 0.009711425, -0.026836876, 0.009291252, 0.009020172, 0.04643593, 0.06966744, 0.03393916, -0.0062788804, -0.077366106, 0.028273597, 0.021008667, -0.0018179273, 0.024885103, -0.0037476749, 0.029086838, -0.018053899, -0.02153727, 0.013553978, -0.0001253743, -0.010816074, -0.026633566, -0.037056576, 0.0071022846, 0.013872497, 0.047981083, -0.040092666, -0.0029700154, -0.03860173, 0.0047405036, -0.025020644, -0.03355965, -0.067336164, -0.045297395, -0.025305277, -0.0039509847, 0.033505432, 0.016264774, -0.011405673, 0.028138058, -0.038683053, -0.024573362, -0.00079926115, -0.00320721, -0.040092666, -0.019029785, -0.03130969, 0.02889708, 0.010328131, -0.07254089, -0.027026633, -0.00826115, -0.02687754, -0.04762868, -0.026010083, -0.036378875, 0.07601071, 0.013818281, -0.036487307, 0.01010449, 0.022770682, 0.04323719, -0.01309992, -0.024898658, -0.037381873, -0.0048218276, 0.024885103, -0.043725133, 0.023028208, 0.026159177, -0.05768573, -0.007420803, 0.024288729, 0.011317572, -0.0041949563, 0.032041606, -0.023326397, -0.01789125, -0.030740421, -0.0046151294, 0.029032622, -0.028165165, 0.01591237, 0.017118674, 0.016440975, 0.033180136, 0.024193851, -0.040146884, -0.014191015, -0.03393916, -0.00283617, -0.041204095, 0.044240184, 0.019138217, -0.043833565, -0.020846019, 0.032339793, 0.03448132, -0.04445705, 0.010626319, 0.057197787, -0.018690936, -0.01949062, 0.011785184, 0.034128916, 0.011805515, 0.02020898, 0.0012495073, 0.037192117, -0.039442077, 0.046923872, -0.057523083, -0.0293037, 0.01100583, 0.028869973, -0.008132387, -0.03461686, -0.015492197, -0.0088439705, -0.066794, -0.05158644, -0.04863167, -0.02800252, 0.0019009454, 0.01120914, 0.0065160748, -0.015803939, -0.0068074856, -0.041041445, -0.015993694, 0.012090148, 0.026714891, -0.028300706, -0.0031326632, 0.01649519, 0.057631515, 0.0011334514, -0.035050586, -0.029737428, 0.038222216, 0.03301749, -0.027839871, 0.0039408193, 0.0038323873, 0.0064042546, 0.035701178, -0.012273127, -0.032854844, 0.014421432, -0.027365481, -0.00596375, -0.042532384, 0.0004400807, 0.0221472, -0.004801497, 0.042125765, -0.010097713, -0.05128825, 0.02023609, -0.005879038, 0.047411814, 0.0005150512, 0.07167344, 0.00722427, 0.01339133, 0.027921194, -0.010436563, -0.041692037, 0.022106538, 0.06408321, -0.026389595, 0.047059413, 0.024763118, 0.022323402, -0.015614183, -0.02136107, -0.0490654, -0.032529548, -0.030957285, -0.012889833, -0.011114262, 0.017593063, -0.06261938, -0.02439716, 0.014868714, 0.043725133, 0.036405984, 0.020425845, 0.006112844, -0.043752242, 0.02201166, 0.019368634, 0.0007696118, 0.02266225, -0.010172261, 0.0104026785, -0.019477066, 0.025942314, -0.027216388, -0.043779347, 0.034237348, -0.0005120862, -0.019856578, -0.00637037, -0.013181244, -0.03274641, -0.03206871, -0.022743575, -0.02439716, -0.03261087, -0.043372728, -0.04998707, -0.052968945, -0.007197162, -0.0021110321, -0.03299038, -0.00895918, -0.043426946, -0.0340747, -0.008830417, 0.045947984, -0.033098813, 0.044104643, -0.047493137, -0.022797791, 0.010639872, -0.0009199763, 0.03022537, -0.033722296, -0.014828051, -0.014028368, -0.002727738, -0.03594515, 0.04136674, -0.0012774625, -0.009765641, -0.040905904, 0.029357916, 0.03483372, 0.041827574, 0.007515681, -0.04348116, 0.012517098, 0.044267293, -0.052860513, -0.038249325, -0.055788174, 0.010328131, 0.0028886916, -0.03358676, 0.025752557, -0.018013237, -0.02678266, -0.021970998, 0.026945308, 0.0036629625, -0.02133396, -0.02313664, 0.017159335, 0.053240027, -0.008369582, 0.037463196, -0.016359651, -0.008274703, -0.02115776, 0.0055368, -0.01526178, 0.012347674, -0.00862033, -0.035077695, 0.014462095, -0.0064754128, -0.036948144, 0.038927026, 0.0151262395, 0.037869815, 0.03266509, -0.06825783, -0.02981875, 0.032963276, 0.05080031, -0.0780167, 0.044267293, -0.057252005, -0.025644125, -0.045785338, -0.017864143, 0.004791331, 0.013127028, 0.00047142428, 0.01843341, 0.036948144, -0.019233095, -0.026457366, -0.0081188325, 0.021821905, -0.0077528753, 0.020480061, -0.010314577, 0.024668239, -0.021821905, -0.050312366, 0.006214499, -0.032692194, 0.02327218, 0.05779416, -0.019002678, 0.0088981865, -0.052182816, -0.035890933, -0.038927026, -0.0014748422, -0.00763089, -0.0031817963, 0.027148617, -0.029872967, 0.0150720235, -0.027040185, -0.019273756, 0.019409297, -0.013736957, -0.023190856, 0.049688883, 0.04627328, -0.010849959, -0.0221472, 0.019585498, 0.01771505, 0.014733174, -0.013147359, -0.010253584, 0.018541843, 0.012571314, -0.03808668, -0.02439716, 0.0066041755, -0.028626002, -0.015112685, 0.040390853, -0.019151771, 0.045134746, 0.021184867, 0.01672561, -0.020263197, -0.03673128, -0.041719146, -0.02865311, -0.027921194, -0.014678958, 0.015858155, 0.053782184, 0.049011186, 0.024559809, 0.08473947, 0.0046964535, 0.032854844, -0.054866504, 0.05730622, -0.006255161, 0.008234042, 0.03621623, 0.019178879, 0.066522926, -0.01699669, -0.006624507, -0.026863985, -0.0007255614, -0.0150991315, -0.014841606, 0.037327655, -0.04643593, 0.012849171, -0.02773144, -0.004255949, 0.0347524, 0.020276751, 0.016346097, -0.005408037, -0.027080849, -0.008803309, 0.01856895, -0.008132387, -0.0206156, 0.012652638, -0.0005247931, -0.03976737, -0.011798738, -0.02245894, -0.015749723, -0.032177143, -0.028300706, 0.007908747, -0.0398758, -0.0236788, -0.023014655, -0.033857837, 0.048306376, -0.041719146, -0.034264456, 0.04177336, -0.014855159, 0.05649298, 0.0013477737, -0.0090066185, -0.007027738, -0.012435775, -0.03130969, -0.023041762, 0.018013237, 0.04350827, -0.014665404, 0.031851847, 0.023163749, 0.04350827, 0.008606776, 0.017728603, -0.041854683, -0.011954608, 0.00058409176, 0.06706508, -0.017064458, 0.04483656, -0.016210558, 0.042423952, 0.018284317, -0.017091567, 0.017728603, -0.0042661144, 0.08701654, 0.040363748, -0.0130321495, -0.0010046887, -0.044619694, -0.013716626, -0.043264296, 0.0551918, -0.015519305, 0.04442994, -0.0033156418, -0.027473914, 0.0012063041, 0.0049776984, 0.0025515363, 0.043969106, -0.027866978, -0.011697083, 0.021686364, 0.03843908, 0.061643492, 0.04456548, 0.012645861, -0.03526745, 0.043860674, -0.028626002, -0.057089355, -0.029222377, -0.0016781519, -0.012849171, 0.00082086277, 0.0029835694, -0.0037307325, 0.001165642, 0.0041814023, 0.009392907, 0.013289675, -0.02664712, 0.025115522, 0.013289675, -0.031743415, -0.01764728, -0.04491788, -0.048604567, -0.036785495, 0.015681952, -0.053890616, -0.012855948, 0.030333802, -0.036812603, 0.0020110714, -0.01379795, -0.011060046, -0.005475807, -0.007332702, -0.010999053, 0.008098502, 0.006038297, 0.02892419, 0.007847753, -0.007237824, 0.010924506, 0.000869996, -0.024451377, 0.0038357757, -0.0048692664, -0.0010580574, 0.02889708, -0.033884946, -0.0062043336, 0.0065160748, -0.025915205, -0.01492293, 0.00032127163, -0.006004412, -0.0028886916, -0.019002678, 0.021523718, 0.0052521667, -0.025576357, 0.0044999206, -0.0027091014, 0.03618912, 0.015871707, -0.03445421, -0.030875962, -0.000740386, -0.010355239, 0.0013740346, -0.018460518, -0.023285734, 0.008918517, -0.037300546, 0.009365799, -0.006004412, -0.002241489, -0.022309847, 0.007183608, 0.0010851653, -0.030577775, 0.0055672964, -0.026701337, 0.042911895, -0.0071090613, -0.01343877, 0.028869973, -0.047276273, 0.029032622, 2.0132422e-05, -0.02572545, -0.0020466507, -0.015681952, -0.03938786, -0.015817493, -0.0034189909, -0.008633884, -0.03862884, -0.015492197, -0.0077867606, -0.01696958, 0.025495032, -0.049770206, 0.04272214, 0.03553853, 0.010111268, -0.03220425, -0.045161854, -0.036297552, -0.048387703, 0.009691094, -0.021862566, -0.012035932, 0.008511898, 0.056547195, -0.032421116, -0.018555395, 0.012483214, 0.018867137, -0.0007175137, -0.012368005, 0.010633096, -0.011243025, -0.038276434, 0.034047592, -0.008782978, -0.0088439705, 0.030631991, 0.028734433, 0.00082552194, 0.0012215523, 0.02981875, 0.017227106, 0.021279745, 0.03805957, -0.029141054, 0.029249484, 0.03431867, -0.0008522064, -0.054947827, 0.006705831, -0.006343262, -0.024315836, 0.013431992, 0.004987864, -0.02156438, -0.008301811, -0.01955839, 0.032936167, 0.010951614, -0.021401731, -0.027568791, 0.00465918, -0.006888809, 0.007800314, -0.03540299, -0.0010038415, -0.0008784672, -0.0063093766, -0.03580961, 0.004831993, 0.07422158, 0.034264456, 0.032529548, 0.03580961, 0.017593063, -0.043779347, 0.05833632, -0.012496768, 0.016129233, -0.009142159, -0.0025650903, -0.013289675, -0.028544677, 0.010023166, 0.023868555, 0.0048387703, 0.017552402, -0.04575823, -0.019477066, -0.0018145388, -0.011595428, 0.048197947, -3.949714e-05, 0.031065717, -0.0059298654, -0.015437981, 0.029710319, -0.0149906995, -0.021550825, -0.027717885, -0.025332386, 0.010341685, 0.040797472, 0.0040594162, -0.0062924344, -0.008599999, 0.048469026, 0.0047540576, -0.030279586, -0.057902593, -0.0006332249, 0.00034880315, 0.027839871, 0.013282899, -0.012971157, -0.022716466, -0.0012029156, -0.018758705, 0.008471237, 0.014543419, 0.02892419, -0.0137843955, -0.00974531, -0.021699918, 0.007827423, -0.030577775, -0.012117256, -0.056221902, -0.019246649, -0.0102129225, -0.0044490932, -0.03220425, -0.022472495, -0.038303543, -0.0012706855, 0.0097317565, -0.008593222, -0.05413459, -0.01400126, 0.0018568949, 0.004913317, -0.021849012, 0.00029077518, -0.017471077, -0.032041606, 0.043562487, 0.03103861, 0.0017772653, -0.01122947, 0.0077122133, 0.031499445, -0.020832464, -0.0017137311, -0.002151694, -0.036541525, 0.01577683, 0.011900392, -0.003995035, 0.0029632384, 0.061697707, 0.01136501, 0.013282899, -0.015383765, -0.009352244, 0.01789125, -0.009826634, 0.035484314, -0.015871707, 0.020005671, -0.0316892, -0.031770524, -0.02426162, 0.022364063, -0.012137587, -0.002953073, -0.010436563, 0.0006844759, 0.0032986994, 0.01931442, 0.047845542, 0.02095445, -0.008694877, 0.012523876, -0.03220425, 0.014394324, -0.002554925, -0.01289661, 0.0059434194, -0.008911741, -0.047032304, 0.0074953497, -0.005699448, -0.010924506, 0.00028759846, -0.04771, -0.009040504, 0.00839669, 0.022133647, -0.020385183, 0.015573521, 0.0043948772, 0.028436245, -0.07503482, -0.02405831, -0.00596375, -0.01724066, 0.03223136, -0.045297395, 0.043264296, -0.01921954, -0.017321983, -0.015858155, -0.013269344, -0.011527658, 0.021754134, 0.006400866, -0.019815916, -0.062402513, 0.03805957, 0.011270133, -0.016142787, -0.02293333, -0.051532224, 0.014014813, -0.019382188, 0.01100583, -0.048089515, 0.040201098, 0.0001939913, 0.013337114, 0.03263798, 0.0033478325, 0.022296295, 0.021252638, 0.036270443, 0.022526711, -0.021930337, -0.001302029, 0.0007086189, -0.021998106, -0.006329708, 0.020602047, 0.036921035, -0.024342945, 0.03895413, -0.009359022, 0.0033020878, -0.043291405, -0.033640973, -0.0015129627, 0.022973992, -0.0150720235, -0.0063500386, -0.00510985, 0.0010199369, -0.0049302597, 0.021862566, 0.04405043, -0.0007831658, 0.01744397, 0.019788807, 0.02851757, -0.013635302, 0.0024837665, 0.030144047, -0.03778849, 0.030252479, -0.0514509, 0.0040661935, 0.0038561067, -0.023353504, -0.018040344, -0.018745152, -0.0029937348, 0.0051708426, 0.017701495, 0.0040526395, 0.02296044, 0.01764728, -0.04217998, -0.021903228, 0.007264932, 0.028544677, 0.013933489, -0.04909251, 0.03727344, 0.0013731874, 0.02518329, -0.023895662, -0.01156832, 0.01901623, 0.000897951, 0.049553342, -0.0010817769, -0.0087897545, 0.0016501968, -0.049363587, -0.032421116, -0.008369582, 0.0051640654, 0.03076753, -0.024302283, 0.031119933, 0.0049234824, -0.018324979, -0.009772418, -0.0068074856, 0.010626319, 0.0070345146, 0.006858313, -0.019775255, 0.003632466, 0.025874544, 0.015668398, -0.012191803, 0.030821746, -0.014014813, 0.020154765, 0.031336796, 0.0055876276, -0.0135946395, -0.016291881, 0.009162489, 0.028056735, 0.010273916, 0.045432933, -0.012713632, -0.00042292647, -0.012049487, 0.02678266, 0.0017586286, -0.022391172, -0.017457524, 0.008267927, 0.001739992, 0.002659968, -0.008342474, 0.041176986, 0.019151771, 0.026416702, -0.057577297, 0.025711896, 0.0070955073, 0.016888257, 0.0074140257, 0.029656103, -0.0158446, 0.011974939, -0.017132228, 0.008044286, -0.026769107, 0.025684789, 0.035728287, 7.1687835e-05, -0.023285734, 0.038547512, -0.03499637, 0.043779347, -0.01699669, -0.017864143, -0.0039442074, 0.03450843, 0.05134247, 0.048956968, 0.009772418, -0.010368793, 0.0017103426, -0.00700063, -0.036649957, 0.012483214, 0.0044762013, 0.0103484625, -0.009867296, -0.02273002, 0.0044050426, 0.03898124, 0.010077382, 0.029141054, 0.026769107, 0.024857996, -0.03499637, 0.030360911, 0.0103213545, 0.014855159, -0.016305435, 0.024315836, 0.031228365, -0.012374782, -0.015817493, 0.008328919, -0.030875962, -0.010409455, -0.0069667445, -0.0070209606, 0.045134746, 0.03391205, -0.00078147155, -0.0028005906, 0.015139793, 0.03431867, -0.008986288, 0.00925059, 0.016739164, 0.008213711, -0.00028696313, 0.013052481, -0.0011851259, 0.010362016, -0.007908747, 0.002885303, 0.02660646, -0.009738533, -0.018731598, -0.031011501, -0.009779195, -0.0021313631, -0.021144206, -0.033640973, 0.01710512, 0.014326555, -0.036107797, 0.00960977, -0.02493932, -0.025237506, 0.052101493, 0.011602205, 0.017823482, -0.010517887, 0.014746728, 0.008044286, -0.051206928, 0.016684948, -0.015749723, -0.0016595152, 0.057143573, -0.004706619, 0.029520564, 0.015275333, -0.009277698, -0.032258466, 0.017023796, 0.017091567, 0.008972733, 0.023380611, 0.003268203, -0.009921512, -0.016603623, -0.0035646963, 0.014245231, 0.026389595, -0.008891409, -0.024017649, 0.00038777085, 0.023231518, 0.03635177, -0.020574939, -0.009223482, 0.00528944, 0.000818745, 0.008112056, 0.03022537, 0.008728762, 0.00012812745, -0.009420015, -0.013086366, -0.014028368, 0.01730843, -0.018975569, -0.003164854, 0.0053436556, 0.022160754, -0.0059197, -0.045649797, 0.0030411738, -0.011581874, 0.01836564, -0.022852007, -0.0042118984, -0.005336879, -0.015749723, 0.010165484, -0.0035579193, -0.0002217346, -0.0022584316, -0.012164695, -0.0070345146, -0.021686364, 0.023570368, -0.017457524, -0.013791173, 0.026511582, 0.008478013, 0.012354451, 0.04307454, -0.0061433404, 0.010477225, -0.028707325, 0.011039715, -0.022675805, 0.011541212, -0.009833411, -0.011148147, 0.010721196, -0.02752813, -0.010924506, 0.0063059884, 0.01672561, 0.032394007, 0.011602205, -0.030198263, -0.03220425, 0.00053495856, 0.013567532, 0.0041271863, -0.006648226, 0.04724917, 0.01170386, -0.038655944, -0.0083356965, -0.0048658783, 0.03450843, -0.010863514, -0.0078070913, -3.3567274e-05, -0.0020330967, 0.01352687, -0.0060823476, -0.00637037, 0.022377618, 0.012279904, -0.02273002, 0.029276593, 0.0062924344, 0.0007344562, 0.017850589, -0.0012427303, 0.0016781519, 0.017660834, -0.024546254, -0.017972576, -0.036649957, -0.03846619, -0.031445228, 0.023041762, -0.00051420403, -0.0005404649, -0.0150449155, 0.024952874, 0.003639243, 0.030062724, 0.01969393, -0.005655397, 0.054595422, -0.022580927, -0.01057888, -0.015681952, 0.011453112, 0.022784237, -0.009115051, -0.017972576, -0.035511423, -0.0041237976, 0.012388336, 0.052697867, -0.025291722, -0.012652638, 0.020778248, -0.008539006, 0.0150178075, -0.0028276986, 0.019165324, -0.028544677, 0.010233253, 0.030469343, -0.0013842, -0.014516311, -0.011602205, 0.014204569, -0.01904334, -0.026077854, 0.0011935972, 0.001151241, -0.036270443, -0.013079588, -0.0067397156, 0.017213551, 0.016088571, -0.0071903854, -0.009697871, 0.013492985, 0.0009267532, 0.029222377, -0.014665404, -0.04217998, -0.0029835694, -0.00540126, 0.020602047, 0.017213551, -0.003117415, -0.00729204, -0.010287469, -0.029195268, 0.035701178, 0.01856895, 0.019504175, -0.015993694, 0.029276593, -0.014489203, 0.004340661, 0.01282884, 0.012320566, 0.024966428, -0.041827574, 0.008742316, -0.019870132, 0.0197617, 0.016603623, -0.04985153, -0.022025215, 0.015465089, 0.01505847, 0.011690306, -0.027663669, 0.016020803, 0.0009835105, 0.015681952, 0.010158706, 0.010484002, -0.007366587, 0.0017450746, -0.0015002559, 0.01246966, -0.014299447, 0.006722773, -0.027189279, -0.008525453, 0.027568791, 0.010334908, 0.0095691085, 0.026416702, -0.024898658, 0.0077935373, -0.011615759, -0.0006455082, 0.0027802598, 0.03716501, -0.0058349874, -0.0109922765, 0.023936326, -0.00074928085, 0.016129233, 0.0039713155, 0.0046354607, -0.010680535, -0.03247533, 0.0087626465, -0.013472654, 0.0017687941, -0.041827574, 0.0134591, 0.019205987, 0.0020517334, 0.003934042, 0.036948144, 0.05784838, -0.019327972, -0.01526178, 0.028083842, -0.010687311, 0.013608194, -0.011358233, -0.044890776, 0.028815757, 0.024641132, 0.028110951, -0.034942154, 0.01226635, 0.010328131, -0.01816233, 0.010924506, -0.016102126, 0.008945625, -0.001082624, 0.024166742, 0.013892828, -0.004679511, 0.033505432, 0.012910164, 0.017728603, -0.011439557, -0.008464459, 0.03529456, -0.01253743, -0.012564538, -0.027284158, 0.0045744674, 0.009738533, 0.0055672964, -0.0058078794, -0.0035409767, -0.008186603, 0.04312876, -0.01850118, -0.022892669, 0.023150194, 0.028571786, 0.018921353, -0.018392747, 0.0026633567, -0.0039408193, 0.01343877, 0.014462095, 0.029601889, -0.00938613, -0.035321668, 0.0023041763, 0.001397754, -0.004676122, -0.0340747, 0.006139952, 0.016752716, 0.0110464925, -0.0009953702, -0.040011343, -0.00988085, -0.0005845153, 0.005726556, -0.024505593, 0.004008589, -0.014936483, -0.044348616, -0.014136799, -0.01087029, 0.018731598, 0.00078486005, -0.0027328208, 0.018880691, -0.014082583, -0.0046219067, -0.011581874, 0.0003903122, 0.006902363, 0.011121039, -0.03011694, -0.0027023244, -0.032339793, 0.024722455, 0.004354215, 0.011812292, 0.0026955474, 0.01190717, 0.0150720235, -0.013506539, 0.00068532303, 0.017294876, 0.002246572, 0.0004172084, 0.031092826, -0.022784237, -0.030171154, 0.004899763, 0.00252104, 0.008010401, -0.005845153, -0.020195426, -0.014367216, 0.046137743, -0.024627578, 0.004001812, 0.0028463353, -0.014421432, -0.008450905, 0.01679338, -0.0008284869, 0.008681322, -0.00096995657, -0.05408037, 0.023516152, -0.00981308, -0.037029468, 0.03182474, -0.02115776, -0.0011597122, -0.0012223994, -0.018704489, 0.036677066, 0.010165484, -0.024193851, -0.035321668, 0.0068244277, 0.0028886916, 0.014055476, -0.035565637, -0.013601417, 0.010572103, 0.023177302, 0.0031970446, 0.030848853, -0.025237506, 0.021211976, -0.03673128, 0.0006111997, 0.009128604, 0.003178408, -0.03263798, -0.024342945, 0.02487155, -0.008450905, 0.010273916, -0.008694877, -0.03778849, 0.018108115, -0.015695507, -0.027623007, 0.0152346715, -0.0055368, 0.014299447, 0.047032304, 0.023434827, -0.019666823, -0.021415286, -0.0037985023, -0.032800626, 0.03399338, -0.023855, -0.019924348, 0.022160754, 0.04350827, 0.016806932, -0.0061162324, -0.0032122927, -0.02183546, -0.012279904, -0.014706066, 0.008437351, -0.0022109926, -0.009359022, -0.0063635926, 0.025467925, 0.025982976, -0.0071497234, -0.0150720235, -0.0077257673, 0.0090066185, 0.030930178, -0.04283057, 0.014516311, 0.006543183, 0.0076986593, -0.007339479, -0.011426004, 0.029141054, -0.00047820128, -0.017132228, 0.0049438137, 0.0060552396, -0.0016840817, 0.049499128, 0.003295311, 0.00081112084, 0.006299211, -0.011270133, 0.03583672, -0.00032254233, -0.025467925, 0.005323325, -0.030848853, 0.014421432, 0.023502598, -0.014082583, -0.0095691085, 0.0221472, -0.052996054, -0.006200945, 0.0068142624, -0.0036019697, 0.018514734, -0.027744994, -0.002144917, 0.017294876, -0.010463671, 0.0008666075, 0.007434357, 0.040661935, 0.009982505, -0.021510163, 0.025169738, -0.006743104, -0.00021919324, 0.02426162, 0.03727344, -0.024546254, -0.019666823, -0.005865484, -0.027812762, -0.045297395, -0.045134746, 0.025589911, -0.019151771, -0.00763089, 0.016142787, 0.012185026, -0.01177163, 0.0077596526, -0.007312371, -0.04995996, -0.007644444, 0.0006933707, 0.011595428, 0.048875645, -0.009840188, -0.025440816, -0.022092985, -0.025400154, 0.03618912, 0.01850118, 0.03792403, -0.032583762, -0.011385341, -0.0633784, -0.005421591, 0.013919936, 0.019870132, 0.030848853, 0.01741686, -0.025169738, 0.0109651685, 0.00585193, -0.022621589, 0.01881292, -0.009528447, -0.0049573677, 0.04817084, -0.02586099, 0.021523718, 0.00834925, 0.04058061, -0.038357757, -0.017254215, 0.025671234, -0.013262567], "content_hash": "60c46f3b672c8bd7794b7d953db20efa"}, {"content": "# LLaMA Overview\n\n## What Is LLaMA?\n\nLLaMA (Large Language Model Meta AI) is a family of open-weight language models developed by Meta. First released in 2023, LLaMA models introduced several architectural improvements over GPT-2 and became the foundation for many other models (Mistral, Qwen, etc.). In the dashboard, models labeled \"LLaMA-like\" share this architecture.\n\n## Architectural Differences from GPT-2\n\nLLaMA models use several key innovations:\n\n### RoPE (Rotary Position Embeddings)\nInstead of GPT-2's learned absolute position embeddings, LLaMA uses **Rotary Position Embeddings (RoPE)**. RoPE encodes position information by rotating the query and key vectors in attention. This means:\n- The model can generalize better to different sequence lengths\n- Position information is baked into the attention computation itself\n- Attention patterns may look different from GPT-2 because of how positions are encoded\n\n### RMSNorm Instead of LayerNorm\nLLaMA uses **RMSNorm** (Root Mean Square Normalization) instead of the standard LayerNorm used in GPT-2. RMSNorm is simpler and slightly faster -- it only normalizes the magnitude of the vectors without centering them first.\n\n### SiLU Activation\nWhere GPT-2 uses GELU activation in the MLP, LLaMA uses **SiLU** (Sigmoid Linear Unit, also called \"Swish\"). This is a smooth activation function that tends to produce slightly different MLP behavior.\n\n### Grouped-Query Attention (GQA)\nLarger LLaMA variants use **Grouped-Query Attention**, where multiple query heads share the same key and value heads. This reduces memory usage and speeds up inference without significantly hurting quality. This means the number of key/value heads may be smaller than the number of query heads.\n\n## Models Using LLaMA Architecture", "source_file": "llama_overview.md", "chunk_index": 0, "embedding": [0.015909318, 0.029880963, 0.038702477, -0.03258351, 0.0003157085, -0.0195807, -0.041507002, -0.005915003, -0.040487174, 0.01974642, -0.023685507, -0.038294546, -0.03054385, 0.0045701046, -0.021148685, -0.024692588, -0.025992868, -0.024934797, -0.013652949, 0.031716652, 0.010727317, 0.0005051336, -0.009376044, 0.0040474427, 0.012690485, -0.020434806, -0.029422041, 0.011186239, -0.01904529, 0.00871953, 0.024514118, -0.03492911, -0.04479595, 0.023532534, -0.04693759, 0.011753519, 0.029167084, 0.012212441, -0.0129072, 0.05236817, -0.03969681, -0.03742769, -0.022703923, 0.02591638, -0.031002773, 0.04364864, 0.050685454, 0.0343937, -0.033373874, 0.021378146, -0.022576444, 0.024845561, -0.03630588, 0.007546728, 0.0015775466, -0.034444693, -0.005895881, 0.008745026, 0.016916398, -0.0013393211, 0.0015982619, -0.012378164, 0.01661045, 0.026795983, -0.012900826, -0.061546624, -0.09520095, -0.008847008, -0.0065970127, -0.0066352566, -0.029498527, 0.006472721, -0.0001759801, -0.014239349, -0.0409206, -0.0219518, -0.0013616298, 0.07439646, -0.014022636, 0.020090614, -0.010905786, 0.017515548, 0.006941205, -0.034674156, 0.03640786, -0.0033271892, -0.0034451068, -0.02206653, -0.070164174, -0.015437648, -0.018012714, -0.018905064, -0.006265569, 0.04130304, 0.04492343, 0.00035116347, 0.022461714, 0.010746438, 0.009420662, 0.04183845, 0.009478027, -0.027127428, 0.021964548, -0.0237365, 0.034980103, -0.020269083, 0.028427707, -0.026770487, -0.022104774, -0.014379576, -0.06684973, 0.014328585, -0.024947545, -0.02692346, -0.008770522, -0.014583542, -0.033883788, 0.016075041, 0.0004282481, -0.0152209345, -0.03908491, 0.006491843, 0.046019744, -0.048365347, 0.018216679, 0.008439077, 0.020778997, -0.000832594, -0.018025462, 0.016011301, 0.010587091, 0.03054385, 0.025355475, -0.029753484, -0.05405089, 0.037172735, -0.034674156, -0.031767644, -0.062974386, -0.0020667454, 0.019440472, -0.021097694, -0.018662853, 0.008464573, -0.045407847, -0.0058321417, -0.016852658, -0.021021206, -0.047371015, 0.0019185516, 0.01287533, -0.026337061, -0.044566486, 0.0013010775, -0.024016952, -0.031716652, 0.014366828, 0.019160021, 0.041073576, -0.016890902, 0.058283176, 0.024641596, 0.045178384, -0.017337078, -0.022206757, 0.05017554, -0.003075419, 0.005475202, -0.051322848, -0.059353996, -0.024233665, 0.026999949, -0.009752106, -0.06435115, 0.010051681, -0.031742148, -0.020600528, -0.035388034, 0.002181476, -0.041430514, -0.019542456, 0.008126755, -0.04257782, 0.012384538, -0.05142483, -0.0038275423, -0.014111871, 0.06603387, -0.0009560888, 0.03528605, -0.013079295, 0.054407828, 0.08643043, 0.052674122, 0.02615859, -0.02527899, 0.07740495, 0.0137039395, 0.036560837, -0.027152922, 0.040742133, 0.013971644, 0.047702458, -0.08000551, 0.018560871, 0.028274734, -0.0027487555, -0.022869645, -0.059303004, -0.015628865, 0.018968802, -0.04550983, 0.05231718, -0.034954607, -0.006440852, 0.0055230064, -0.07281572, 0.013474478, 0.009771228, 0.046580646, 0.028274734, 0.007247153, 0.027254906, 0.051654294, -0.058028217, 0.016087787, 0.041557994, -0.060883738, 0.030212408, -0.011931988, 0.0038402902, 0.0055931197, -0.027866803, -0.023354063, 0.0028905752, -0.08122931, 0.032252062, -0.023621768, 0.029804476, 0.0025862204, 0.01643198, 0.009082844, -0.01136471, -0.009624627, 0.03809058, 0.01737532, -0.018140191, -0.03714724, 0.036050923, 0.021926304, 0.015743596, -0.006252821, -0.03179314, -0.008164999, -0.018560871, 0.04739651, -0.0587421, 0.027101932, -0.049742114, -0.010701821, -0.014647281, 0.056804426, -0.0013010775, 0.06363727, 0.002989371, -0.025431963, -0.048926253, 0.0225382, -0.021543868, -0.040206723, 0.01388241, 0.004436252, -0.038473014, 0.003496098, 0.0068583437, -0.024055194, 0.015552378, -0.012951816, 0.019950386, 0.047243536, 0.016980138, -0.048569314, 0.042501334, 0.028249238, -0.0022372478, -0.004949353, 0.020766249, -0.03566849, -0.027917793, -0.005746094, 0.008177746, -0.03237954, 0.04324071, -0.022168513, -0.011995728, 0.016954642, 0.017553791, 0.0040633776, -0.010019811, 0.025393719, 0.031028269, -0.0033271892, -0.03237954, -0.02705094, 0.0055771847, 0.03365433, -0.0048186877, -0.03566849, -0.047676962, 0.025954625, 0.013155783, 0.026005616, 0.016291754, -0.011422074, -0.031181244, -0.03571948, -0.027101932, 0.011575049, 0.031589177, -0.01880308, -0.02651553, -0.03992627, -0.0020444365, -0.05450981, -0.009439784, 0.017834244, 0.0026037486, -0.044056572, -0.04561181, -0.045407847, 0.08581854, 0.03054385, -0.006781857, -0.013614705, -0.024450378, -0.002614903, -0.025584938, 0.02562318, 0.005650485, 0.03505659, -0.038167067, -0.01969543, 0.005210684, 0.033348378, -0.028631674, 0.0052744234, 0.000274278, -0.041073576, 0.028759152, -0.027815811, 0.004085686, -0.040181227, 0.035617497, -0.0018117883, -0.014175611, 0.014609037, 0.020575032, -0.03594894, -0.021709591, -0.000800326, -0.06919534, 0.020766249, -0.02420817, -0.017987218, -0.016113283, -0.00083100057, 0.040895108, 0.009892332, 0.018726593, -0.034240726, 0.020230839, 0.021735087, 0.017171355, 0.023889473, -0.01709487, 0.04247584, 0.0083307205, 0.012779721, -0.025164258, 0.033195402, 0.01886682, 0.011039639, 0.025788903, -0.0034546677, 0.02651553, 0.04579028, -0.014851247, 0.006479095, -0.029269066, 0.018382402, -0.01974642, -0.01026202, -0.04135403, 0.023621768, 0.03393478, 0.017005634, 0.017987218, -0.0012819557, 0.030390877, 0.042220883, 0.037376698, -0.020473048, -0.023647264, -0.0033144413, -0.009248566, -0.0045924135, 0.005672794, 0.026694, 0.0015257584, 0.0038689729, 0.032073595, 0.010529725, -0.0139461495, -0.0343937, 0.0018452514, -0.046580646, 0.025495702, 0.0040633776, 0.06735965, 0.03689228, 0.01631725, -0.040308706, -0.009892332, -0.019682681, -0.006485469, 0.019657187, 0.03890644, -0.026388051, -0.041532498, -0.0462747, -0.03352685, 0.0052553015, 0.033348378, -0.04278179, -0.0048505575, -0.012461024, -0.029727988, 0.019249255, -4.987099e-05, -0.0011855501, 0.050150048, 0.021684095, -0.010191907, 0.023417803, -0.004184482, 0.016266258, -0.047014076, -0.0014221822, -0.021314407, -0.0142011065, 0.01844614, 0.018012714, 0.04900274, -0.00030614762, 0.004350204, -0.06129167, 0.05231718, -0.029829971, 0.025100518, -0.035693984, 0.069399305, -0.021097694, -0.020855485, -0.011562301, -0.001956795, -0.006077538, 0.04038519, -0.017821496, 0.030033937, 0.0061317165, -0.013933402, -0.051858258, 0.005924564, -0.00066049804, -0.05203673, -0.011198987, -0.008942617, -0.037784632, 0.029498527, -0.01719685, 0.011785388, 0.012805216, 0.018994298, 0.0007967407, -0.015705353, 0.0522152, 0.0021209237, 0.015386656, 0.022793157, 0.005370032, -0.025827147, 0.0058130203, 0.05216421, -0.0136911925, 0.024386639, -0.027433375, 0.010682699, -0.056804426, -0.026388051, -0.027713828, -0.01625351, -0.007189788, -0.011077883, 0.055682614, 0.0028507381, 0.019070785, -0.018828576, 0.022550948, 0.0022818653, 0.035158575, 0.009975193, 0.0030897604, -0.009216696, 0.026413547, -0.0004282481, 0.0385495, -0.011944736, 0.042093404, 0.010198281, 0.030722322, 0.046427675, -0.0060169855, -0.04364864, -0.014303089, 0.010682699, -0.016750677, -0.022028286, -0.030824304, -0.021339903, -0.038320042, 0.009847715, 0.002860299, 0.0154249, 0.0035598374, -0.03890644, -0.0032283934, 0.0006051246, -0.03406226, -0.045484334, -0.012033971, 0.018828576, 0.046912093, 0.048492827, -0.0013608331, -0.008885252, -0.005175628, 0.027484367, -0.027535358, 0.019618943, 0.030339886, 0.055325672, -0.00599149, 0.016177023, -0.038065083, 0.016482972, -0.021505624, -0.009541767, -0.004445813, 0.015322917, 0.0004314351, 0.032838464, -0.003843477, 0.006154025, -0.021378146, -0.017872488, 0.0038817206, 0.01183638, 0.014022636, 0.020218091, 0.020230839, -0.017158607, -0.006941205, 0.019287499, 0.011390205, 0.004104808, -0.025495702, 0.031895123, 0.028580682, 0.04194043, 0.018535376, -0.03304243, -0.028988613, -0.01583283, 0.0036745681, -0.0379631, 0.010816552, -0.035846956, 0.011084257, -0.004359765, 0.040971592, 0.04816138, -0.02094472, -0.022041034, -0.0060870987, -0.024195421, -0.04130304, 0.016623197, -0.032659996, -0.032889456, 0.0026483661, -0.002178289, 0.018764837, -0.013359748, -0.0010564781, 0.001502653, 0.016635945, -0.023417803, 0.0048378096, 0.02206653, -0.018752089, -0.023303071, 0.0015106204, -0.018650105, -0.053540975, -0.052827094, 0.004209978, -0.02865717, 0.0019201451, 0.040079243, -0.0551727, -0.008477321, -0.016916398, 0.02799428, -0.019465968, 0.011428448, -0.006711744, -0.035795964, -0.021505624, -0.062056538, 0.036790296, -0.026018364, 0.0050800187, 0.044591982, 0.014379576, 0.024998536, 0.04385261, 0.021543868, -0.030084928, 0.045586314, -0.03000844, 0.027280401, 0.024067942, -0.015450396, -0.03849851, 0.0043151476, -0.044362523, -0.00979035, 0.011065135, 0.025635928, -0.005937312, -0.012817964, -0.034419198, 0.024667092, 0.01441782, 0.021696843, -0.014723768, 0.024781823, -0.009720236, 0.03582146, 0.04619821, -0.071285985, -0.017005634, -0.023570776, -0.019950386, -0.03245603, 0.0385495, 0.0011273881, -0.009012731, 0.02272942, -0.0042195385, -0.000746546, 0.008158625, 0.006915709, 0.0111479955, -0.009541767, -0.016980138, 0.008643043, -0.050455995, -0.027713828, 0.008273356, -0.0073300144, 0.0028682665, -0.05071095, -0.010332134, -0.0026212768, -0.0011720055, 0.001241322, -0.0015759531, -0.0068838396, 0.013614705, -0.014315837, -0.040793125, -0.0046689003, -0.035490017, 0.0018707472, 0.0008381712, -0.0074128755, 0.05150132, -0.009707489, -0.038039587, -0.033552345, -0.008273356, -0.02735689, 0.025240745, -0.0052680494, -0.0116515355, -0.020103361, 0.027790315, 0.025546694, -0.00670537, 0.005660046, -0.014813003, -0.0031646541, -0.00054736086, 0.010924908, -0.0052935453, 0.009828594, -0.0076805805, -0.006485469, -0.00029798102, -0.035744976, 0.0076614586, -0.030951783, 0.0063484297, 0.0130538, -0.013665697, -0.06475908, 0.019300247, 0.0031072886, 0.0026961705, -0.0011759893, 0.022295991, 0.03944185, -0.004955727, -0.027968785, -0.007438371, -0.013895158, -0.004907923, -0.011179865, -0.00041669537, 0.013576461, 0.002218126, -0.032558013, 0.017349826, 0.04194043, -0.018025462, 0.024654344, 0.0052584885, 0.023953212, -0.02830023, -0.054968733, 0.009662871, 0.037631657, 0.011734397, -0.0078144325, -0.05088942, 0.019389482, 0.0073236404, -0.021798825, 0.003349498, 0.0030371756, -0.008375338, 0.0009975194, 0.00097202364, 0.0045223003, -0.031002773, 0.0018707472, -0.0070623094, 0.027433375, 0.009803098, -0.013397992, 0.0228314, 0.03630588, 0.012856208, -0.0019870712, -0.009070096, 0.07862875, 0.010427742, -0.0033718066, 0.025648676, -0.007431997, -0.008623921, 0.049589142, -0.042628814, 0.011976606, 0.0046752742, -0.029702494, -0.05384692, -0.0013990767, -0.013499974, -0.0308498, -0.005115075, 0.016380988, -0.01346173, 0.020065118, 0.030518355, 0.021161433, -0.08862306, -0.04443901, 0.017757757, -0.024233665, 0.017897982, 0.020192595, 0.013614705, -0.010096298, 0.03760616, -0.057569295, 0.032303054, -0.04364864, 0.008515565, -0.004907923, 0.00089633325, -0.05384692, -0.007024066, 0.011058761, 0.05486675, -0.0021990044, -0.02437389, 0.018344158, 0.0026738618, 0.0053413496, 0.013423488, 0.002350385, 0.015552378, 0.03418974, 0.04109907, 0.0037988597, -0.022563696, -0.004904736, -0.019771917, 0.021786077, -0.014264845, -0.02088098, 0.021645851, -0.023660012, -0.0030674518, -0.037912108, -0.0056249895, -0.021671347, 0.02343055, -0.010759186, -0.00857293, 0.01820393, 0.043087736, 0.011728023, -0.012461024, -0.0009807878, 0.008541061, 0.020358318, -0.007795311, 0.021786077, 0.023812985, -0.010791056, -0.015998553, -0.01625351, 0.016406484, 0.029982945, -0.032252062, 0.02628607, -0.007266275, -0.006424917, -0.03867698, -0.013640201, -0.0064536, 0.024259161, -0.019172767, -0.040971592, -0.019185515, 0.03232855, -0.008337094, 0.026974453, 0.020778997, -0.019657187, 0.013283261, 0.052878086, -0.008145877, -0.018586367, 0.024259161, -0.03949284, 0.0063834866, -0.012970938, 0.012199693, 0.02924357, 0.017796, -0.030798808, -0.013729435, 0.035235062, -0.006485469, -0.009050975, 0.03982429, 0.022092026, 0.0014094342, -0.0023758807, 0.0017368947, 0.02320109, 0.008802392, -0.013856914, 0.021480128, 0.010848422, -0.038396526, -0.036280382, 0.017413564, -0.015692605, 0.0015520508, 0.001797447, 0.014914986, 0.019835657, 0.04854382, 0.032277558, -0.0010923315, 0.026362557, -0.018433392, 0.012671364, 0.05960895, 0.010160037, 0.05083843, 0.008343468, 0.011205361, -0.014519802, -0.027152922, -0.009917828, 0.002138452, -0.03747868, 0.013627453, 0.057518303, 0.0012309643, -0.014392324, -0.014978725, 0.008152251, -0.022576444, -0.017745009, -0.06328033, -0.008419956, 0.0154758915, -0.015565126, 0.017655773, 0.015628865, 0.0009728204, -0.0175028, 0.038116075, -0.0068647177, -0.013576461, 0.006947579, -0.022512704, -0.018752089, -0.030977279, -0.010000689, 0.03250702, 0.011422074, -0.008776896, 0.053133044, 0.0009983161, 0.00272804, -0.0013783614, -0.064045206, -0.01130097, 0.014519802, 0.006399421, -0.0014094342, 0.0007106927, -0.004107995, 0.0001411227, 0.008917122, -0.025878137, 0.0121487025, -0.01743906, 0.0014946855, 0.0108930385, -0.025355475, 0.035311546, 0.0385495, 0.009592758, 0.0020906476, 0.01186825, -0.0051883752, 0.0020508105, 0.028274734, -0.025941877, 0.036229394, -0.004209978, 0.040308706, 0.010446864, -0.006520526, -0.0047740703, -0.03125773, -0.0108866645, 0.042934764, 0.0018436579, 0.03375631, -0.01023015, -0.009031853, -0.003610829, -0.008636669, 0.04941067, 0.015437648, -0.021161433, -0.010160037, -0.02123792, 0.10244173, -0.013397992, -0.0058098333, -0.031767644, -0.026362557, -0.004949353, -0.009592758, -0.0013329472, 0.007973781, 0.007004944, -0.01596031, -0.022691175, -0.008177746, 0.0045796656, -0.036917776, -0.016661441, -0.0041621733, -0.02735689, -0.04158349, -0.038320042, -0.01661045, -0.032812968, 0.031155748, -0.004378887, 0.0006760345, -0.033144414, -0.009694741, -0.026541026, -0.0031869628, -0.0052776104, 0.0039582076, 0.0017098056, 0.016916398, -0.03666282, 0.015412152, 0.018407896, -0.020626023, -0.0027232596, 0.00818412, -0.0019010233, -0.0007903668, -0.024832813, -0.0053668455, 0.016470224, -0.009554515, 0.024488622, 0.027637342, -0.022206757, -0.03826905, 0.007763441, 0.023838481, -0.013754931, -0.008942617, 0.0070686834, 0.006485469, 0.033501353, 0.03768265, 0.04216989, 0.00928681, -0.027560854, 0.045433342, 0.0228314, -0.0029447535, 0.010593465, -0.008050268, -0.0014150115, 0.023277577, 0.014647281, 0.005172441, -0.025521198, 0.02817275, 0.024412135, 0.03166566, -0.0011720055, 0.026795983, 0.02705094, -0.0124100335, 0.0063165603, 0.0070686834, -0.01488949, -0.012378164, -0.027101932, 0.0013504755, -0.012002102, 0.017745009, -0.008993609, -0.013245017, 0.054968733, -0.022461714, -0.0074128755, 0.0616996, 0.004146239, -0.005229806, -0.01248652, 0.016240763, -0.010357629, -0.017936226, 0.02112319, -0.024526866, 0.03582146, 0.00391359, 0.0051118885, -0.0114029525, -0.013538218, -0.0056122416, -0.016712433, -0.04002825, 0.015322917, -0.0058703856, -0.02367276, -0.00040673613, 0.014468811, -0.009293184, 0.02017985, -0.008024773, -0.029269066, -0.012288928, -6.891807e-05, -0.022104774, 0.01596031, 0.019121777, -0.002320109, -0.015055212, 0.007527606, 0.015794588, 0.026999949, 0.021569364, -0.006928457, 0.0032331736, 0.028249238, -0.012983686, -0.0020285018, 0.020192595, -0.021021206, -4.7605256e-05, -0.027790315, -0.04199142, -0.0021065823, -0.04586677, 0.025126014, -0.0038084206, -0.015157195, -0.0069029615, -0.0068137264, 0.01785974, 0.012626747, -0.030263398, 0.04064015, -0.008923496, 0.05262313, 0.004394822, -0.011428448, -0.018433392, -0.028963117, 0.053948905, -0.0033176283, 0.010918534, -0.04400558, -0.022308739, 0.0126649905, 0.018956054, -0.01233992, -0.006941205, 0.025431963, -0.0049907835, -0.03258351, 0.033246394, 0.033781804, 0.01086117, -0.0019982257, 0.010574343, -0.010931282, -0.002049217, -0.0023583525, 0.0119128665, 0.00132737, 0.021773329, 0.0010835674, 0.019593447, -0.023315819, -0.007202536, -0.003112069, 0.026795983, -0.010733691, 0.014851247, 0.01661045, 0.0063069994, 0.014379576, -0.0052935453, 0.008713156, 0.0003800453, -0.027866803, 0.00097043015, 0.0055357544, -0.012926321, 0.0030706387, 0.0018054144, -0.028096264, 0.003763803, -0.019363986, -0.034419198, 0.00491111, -0.0032395476, 0.040665645, -0.005462454, 0.008617547, 0.03505659, -0.0061189686, 0.009924202, -0.035974436, 0.043036744, -0.04739651, 0.01933849, -0.01086117, -0.0246161, -0.009057349, 0.021645851, -0.024284657, 0.022691175, -0.032150082, 0.0024157178, 0.017783253, 0.024883805, 0.009452532, 0.006230512, -0.010000689, -0.013512722, 0.011708901, 0.044464506, 0.0041526123, 0.004968475, 0.0015855139, 0.003349498, 0.001287533, -0.0056122416, 0.011740771, -0.041736465, -0.034495685, 0.039416354, -0.00023842465, 0.012365416, 0.025253493, 0.037580665, -0.011256352, 0.031767644, -0.026744992, 0.011626041, 0.02005237, 0.007247153, -0.0034036762, 0.00047087375, -0.04069114, 0.024577856, -0.0028379904, 0.017706765, -0.010281142, -0.004104808, 0.00491111, -0.0071387966, 0.011766267, -0.007202536, 0.024233665, -0.004114369, 0.006176334, 0.023596272, -0.012760599, 0.039135903, 0.00092182896, -0.047141552, 0.01631725, 0.014366828, 0.040487174, 0.013270513, 0.016279006, -0.0053445366, -0.009803098, -0.022015538, -0.00591819, -0.015373908, -0.03299144, 0.012569381, 0.002485831, -0.040869612, 0.02260194, 0.015552378, -0.0060329204, -0.007769815, -0.024807319, 0.027382385, 0.027611846, 0.015947562, 0.0028124948, 0.027433375, -0.016138779, -0.024552362, 0.026744992, 0.0019583886, -0.023761995, -0.024858309, -0.0032698237, -0.031028269, -0.0057970854, 0.028504195, -0.008961739, -0.024335647, 0.011256352, -0.0063771126, 0.0022531827, -0.0175028, -0.022805905, -0.012218815, -0.023570776, 0.01649572, 0.047549486, -0.03385829, -0.020320075, 0.016661441, -0.014009888, -0.021913556, -0.011441196, -0.0013903126, -0.0012054687, 0.02758635, -0.02852969, -0.014264845, -0.031359714, -0.004538235, 0.01565436, 0.011154369, -0.006762735, 0.022270495, -0.019657187, -0.04385261, -0.0052999193, -0.030620338, 0.008177746, 0.0020157539, 0.0032889456, 0.0062113903, 0.027178418, -0.008439077, -0.024016952, -0.0047422005, -0.0016667815, 0.03357784, 0.029269066, 0.02420817, 0.058436148, -0.008770522, 0.013066547, -0.00030694436, -0.0071260487, -0.0068965876, -0.013015556, 0.031538185, 0.021492876, 0.020931972, 0.014124619, 0.02604386, 0.0017878861, 0.004168547, -0.013296009, 0.007846302, 0.015705353, -0.02112319, -0.003508846, -0.008241486, 0.01719685, -0.021747833, 0.016839912, -0.018841324, 0.031946115, 0.0063452427, -0.006679874, 0.015973058, 0.009962446, 0.01047236, 0.0022643371, 0.020371066, 0.0045764786, -0.0057811504, 0.013168531, 0.0073427623, 0.0037829247, -0.0009879585, 0.0023424176, -0.0021065823, -0.009248566, 0.01248652, 0.016929146, 0.03554101, 0.03197161, 0.009509897, -0.018433392, 0.0053572846, 0.024284657, 0.0033048803, -0.03944185, -0.013576461, 0.013499974, 0.0093505485, -0.015769092, -0.026133094, 0.01690365, -0.016419232, -0.025457459, 0.004793192, 0.039059415, 0.033322882, -0.0065269, -0.016151527, 0.012786095, 0.01974642, -0.011957484, 0.076640084, -0.010255646, -0.0025113267, -0.010791056, 0.0010620554, -0.016482972, 0.0028316164, 0.0006955546, -0.014558046, -0.039849784, 0.008088511, -0.008496443, -0.020804493, 0.0021209237, 0.004617909, -0.024182674, 0.012531138, 0.006144464, -0.0198739, 0.0076487106, 0.0035662113, 0.03304243, -0.0152464295, -0.0076359627, 0.019631691, -0.009975193, 0.013805923, -0.01714586, -0.022920636, 0.0071196747, 0.022334235, -0.013869662, -0.017604783, -0.03625489, 0.0326345, -0.016980138, -0.012607625, 0.019618943, 0.041150063, 0.031716652, 0.023634516, -0.013347, 0.014914986, -0.01726059, 0.013844166, 0.014749263, -0.015845578, -0.047116056, 0.013895158, 0.015858326, -0.02011611, -0.006934831, 0.0026531464, -0.005468828, 0.03447019, 0.0027742512, 0.009057349, 0.007852676, 0.0026834228, 0.04721804, -0.000117917625, -0.0060520424, 0.012779721, 0.006966701, 0.004872866, -0.024590604, -0.023851229, 0.016419232, 0.0137039395, -0.03653534, 0.026362557, -0.016865406, 0.013742183, 0.060118865, -0.00720891, -0.03928888, 0.02651553, -0.03286396, -0.011511309, 0.010504229, 0.020154353, 0.025317233, -0.016177023, -0.008496443, -0.010720943, 0.06567693, 0.05096591, 0.010038933, 0.007259901, -0.005424211, 0.010733691, -0.00753398, -0.012282555, -0.018063705, -0.021569364, -0.040614653, 0.0067181177, 0.007686954, 0.027611846, -0.006179521, -0.00269139, 0.015131699, -0.0290651, -0.0071069268, -0.00516288, -0.0063261213, 0.03237954, 0.027688332, -0.013895158, -0.020039622, 0.016623197, 0.008764148, 0.0005736533, 0.008477321, -0.00624326, 0.01898155, -0.015820082, 0.0103767505, 0.0005792305, 0.0139588965, 0.0011162338, 0.0014421006, 0.010752812, 0.0063101863], "content_hash": "a132e8009ac47f488a4966c1492e678a"}, {"content": "Larger LLaMA variants use **Grouped-Query Attention**, where multiple query heads share the same key and value heads. This reduces memory usage and speeds up inference without significantly hurting quality. This means the number of key/value heads may be smaller than the number of query heads. ## Models Using LLaMA Architecture\n\nThe dashboard's \"llama_like\" family includes:\n- **Meta LLaMA**: LLaMA 2 (7B, 13B, 70B), LLaMA 3 (1B, 3B, 8B, 70B)\n- **Qwen**: Qwen2, Qwen2.5 (0.5B to 72B) -- available in the dashboard dropdown as \"Qwen2.5-0.5B\"\n- **Mistral**: Mistral-7B, Mixtral-8x7B\n\n## What to Expect in the Dashboard\n\nWhen using a LLaMA-like model (such as Qwen2.5-0.5B):\n\n- **More layers and heads**: Even the small Qwen2.5-0.5B has 24 layers and 14 heads, compared to GPT-2's 12 layers and 12 heads\n- **Different attention patterns**: RoPE-based attention may show different positional patterns compared to GPT-2\n- **Different tokenizer**: LLaMA-family models use a different BPE vocabulary, so the same text may tokenize differently\n- **Comparing with GPT-2**: Running the same prompt on both GPT-2 and a LLaMA-like model is a great way to see how architecture affects predictions\n\n## HuggingFace Model IDs\n\n- `Qwen/Qwen2.5-0.5B` (in default dropdown)\n- `meta-llama/Llama-3.2-1B`, `meta-llama/Llama-3.1-8B`\n- `mistralai/Mistral-7B-v0.3`", "source_file": "llama_overview.md", "chunk_index": 1, "embedding": [0.02890877, 0.012710809, 0.065590434, -0.030106649, 0.00014588732, -0.005696572, -0.031916775, 0.005363828, -0.023385227, 0.010301745, -0.028216664, -0.031144809, -0.02106933, 0.0074135293, -0.013449499, -0.008378486, -0.010122063, -0.0040395083, -0.010488081, 0.031384382, -0.020749897, 0.014547554, -0.010394913, 0.020616798, 0.015745431, 0.0018101257, -0.04384231, -0.015040015, 0.008638026, 0.03106495, 0.030692277, -0.03630899, -0.028615957, -0.007832786, -0.06330116, 0.019591948, 0.007473423, 0.029068489, -0.026047176, 0.044534415, -0.035111114, -0.044853847, -0.024955776, 0.003746694, -0.011080365, 0.0178883, 0.01317665, 0.041819226, -0.044108503, 0.039290376, -0.0076198303, 0.034951396, -0.05792402, 0.016410919, 0.0044421284, -0.03803926, -0.025434928, 0.0056300233, 0.031277906, 0.002623684, -0.01052801, -0.006069245, 0.019552018, 0.031836916, 0.01893977, -0.05840317, -0.070328705, -0.014321288, 0.0017835061, -0.005696572, -0.025568025, 0.024343528, -0.00047374383, 0.009217, -0.012870526, 0.008498274, 0.0009341779, 0.042431474, -0.012737428, -0.0005677439, 0.0069210688, -0.0041260216, 0.011858985, -0.04267105, 0.0273116, -0.012165109, -0.01796816, -0.017568866, -0.05262674, -0.040834304, -0.016756972, -0.023318678, -0.025780981, -0.001317665, 0.025102183, 0.016623875, 0.016118104, -0.010707691, -0.018780053, 0.04048825, 0.026486397, 0.0024373475, 0.011765816, -0.02259995, 0.027218433, -0.0294944, 0.021375455, -0.020656727, -0.054995876, 0.023864377, -0.0527066, 0.012777357, -0.042431474, -0.00845169, 0.0015522493, 0.014547554, -0.055102356, 0.0016296122, -0.0010381603, -0.0247162, -0.017475698, -0.006847865, 0.058669366, -0.03907742, 0.0061324663, 0.0017469043, 0.020310674, 0.005546837, 0.01846062, -0.0044521103, 0.008305282, 0.02160172, 0.034578722, 0.0029813834, -0.013243198, 0.029068489, -0.019219276, -0.01953871, -0.067879714, -0.04594525, 0.0091637615, -0.025993936, 0.008398451, 0.008278663, -0.04264443, 0.03362042, -0.01738253, -0.027418079, -0.061544273, 0.015945077, 0.0088776015, -0.01900632, -0.04690355, -0.016517397, -0.035749983, -0.033460706, 0.011965463, -0.024183812, 0.022520093, -0.045519337, 0.025421618, 0.0024373475, 0.036495328, -0.023505013, 0.005939475, 0.051402245, -0.013802208, 0.008032433, -0.066016346, -0.056593046, -0.0050610313, 0.039290376, 0.015638953, -0.08587449, -0.0056666248, -0.031224668, -0.020443771, -0.028695816, -0.008531548, -0.015505856, -0.023105722, -0.0050077923, -0.011479657, 0.010188611, -0.0484741, -0.007433494, -0.016743662, 0.04163289, 0.011779127, 0.018008089, 0.0012902137, 0.045971867, 0.09540427, 0.034152813, 0.010488081, -0.022014322, 0.071233764, 0.031304527, 0.0054603238, -0.0051908013, 0.006861175, 0.01842069, 0.06889125, -0.0778354, 0.016610565, 0.02896201, -0.0294944, -0.028003708, -0.054569963, -0.029308064, 0.011872294, -0.029361302, 0.05169506, -0.03588308, -0.055687983, 0.021535171, -0.08906883, 0.0315441, 0.010641143, 0.047888473, 0.017502317, 0.01944554, 0.03580322, 0.030186508, -0.067507036, 0.023837758, 0.010135372, -0.041579653, -0.015718812, 0.0050543766, 0.01265757, -0.0007470096, -0.036708284, 0.0077263084, -0.021867914, -0.06660198, 0.0152929, -0.034019716, 0.009696151, -0.028669195, -0.00047665532, 0.007526662, -0.00973608, 0.006811263, 0.038678125, 0.014787129, -0.021894535, -0.07309713, 0.04727622, 0.005476961, 0.019871453, -0.01346281, -0.047648896, 0.009569708, -0.02371797, 0.046557497, -0.06889125, 0.046131585, -0.03625575, -0.01685014, -0.017023167, 0.027524557, -0.023598183, 0.081242695, -0.0031860208, -0.03521759, -0.025568025, 0.026180273, -0.046131585, -0.045066804, -0.0042790836, 0.0058862357, -0.024143882, 0.020803135, 0.0051442175, -0.02318558, 0.008864292, -0.027418079, -0.0073536355, 0.011093674, 0.0058163595, -0.032076493, 0.03409957, 0.016064866, -0.005849634, -0.0022776306, 0.04264443, -0.031144809, -0.014600793, 0.0049179518, 0.0037799682, -0.024662962, 0.053984337, -0.015612334, -0.0062722187, 0.008431725, -0.006228962, -0.031916775, -0.006012678, 0.014454385, 0.03625575, -0.01294373, -0.028562717, -0.024609722, 0.016583946, 0.0273116, 0.013429535, -0.0022643209, -0.06644226, 0.0080257775, -0.017049786, 0.023132341, 0.00871123, 0.01477382, -0.014361218, -0.054090813, -0.019365683, 0.025208661, 0.033354226, -0.015332829, 0.0015688865, -0.043655973, -0.016264511, -0.028003708, 0.010354984, -0.0011621073, -0.014534244, -0.031357765, -0.041020643, -0.04903311, 0.06005358, 0.024609722, -0.0014266386, 0.019858142, -0.039396852, -0.01686345, -0.010601214, 0.024370147, -0.0075200074, 0.043975405, -0.035909697, 0.007473423, -0.010062169, 0.019086178, -0.049831696, -0.012477888, 0.02317227, -0.0003940933, 0.0274447, -0.033487324, 0.014121642, -0.026459778, 0.0076131755, -0.045971867, 0.0027900557, 0.044374697, 0.03484492, -0.02528852, -0.028935391, 0.009995621, -0.02528852, 0.014560863, -0.027977088, -0.01368242, -0.0035936318, 0.006079227, 0.041925706, 0.0015522493, 0.036495328, -0.031357765, 0.016690424, -0.014387837, 0.01634437, -0.02584753, -0.015479236, 0.0047415975, -0.009662877, 0.039982483, -0.02890877, 0.031171428, 0.0077196537, 0.0031344455, 0.022027632, 0.0041659507, 0.03684138, 0.038864464, 0.0026087104, 0.016490776, -0.04490709, -0.019126106, -0.0020696658, -0.0015439307, -0.023225509, -0.0020014532, -0.0006197351, 0.010933957, 0.038092498, 0.013549323, 0.022586642, 0.037560105, 0.026033865, -0.015625643, -0.029015249, -0.011819055, 0.0010631161, 0.008325247, -0.010301745, 0.013868757, 0.019259205, -0.016158033, 0.035457168, -0.0019898072, -0.0111535685, -0.0189797, -0.016171344, -0.028828913, -0.00038078355, 0.020430462, 0.06905097, 0.031251285, -0.009622947, -0.014840368, -0.015705502, 0.016357679, -0.017355911, -0.013542668, 0.03793278, -0.056113895, -0.017009858, -0.030053409, -0.036495328, 0.007852751, 0.042990483, -0.033753518, -0.008338557, -0.019711737, -0.014294669, 0.021109259, 0.0051908013, -0.016410919, 0.038278833, 0.012531127, 0.0102218855, 0.033993095, -0.012837252, 0.0035304106, -0.036655046, -0.045412857, -0.017821752, -0.043017104, 0.007999158, 0.030266365, 0.034685202, 0.013223234, 0.001683683, -0.06740056, 0.07272446, -0.017276052, 0.025355069, -0.033913236, 0.06207666, -0.0099623455, -0.0119787725, 0.025860839, -0.01895308, -0.002818339, 0.057072196, -0.01186564, 0.029281445, -0.020310674, -0.00053197396, -0.049778454, 0.0115661705, -0.0025005687, -0.04991155, -0.008378486, -0.008491619, -0.015652264, 0.038731366, -0.03585646, 0.014481005, -0.0025654538, 0.030905234, 0.018034708, -0.010574594, 0.06798619, -0.0008514079, -0.004555261, 0.039663047, 0.005743156, -0.035457168, -0.007573246, 0.03793278, -0.014813749, 0.026060486, -0.015066634, 0.0020630108, -0.035084493, -0.011712578, -0.0390508, -0.011805746, 0.007420184, -0.0029148348, 0.046584114, 0.0009965673, -0.001624621, 0.0041393316, 0.0039862692, 0.0062988377, 0.023505013, 0.0065417406, -0.005027757, -0.016996548, 0.034924775, -0.005896218, 0.050789997, -0.007806167, 0.028536098, 0.013509394, 0.050710138, 0.011719232, -0.014015164, -0.037214052, -0.030266365, -0.00030924365, -0.028642576, -0.034685202, -0.057231914, -0.007626485, -0.03569674, 0.031224668, -0.01685014, -0.0080257775, -0.01685014, -0.07235178, 0.0115395505, 0.027631035, -0.046584114, -0.016144723, -0.00044754025, 0.02684576, 0.053558424, 0.021189118, 0.0028998612, 0.0036435432, 0.003144428, 0.017741892, -0.031304527, 0.024396766, 0.03428591, 0.050976332, -0.023784518, 0.024649652, -0.0018101257, 0.013090136, -0.029680736, -0.008611406, 0.009975656, -0.0072937417, -0.010521355, 0.05214759, -0.008298627, 0.0028249938, -0.03324775, -0.02373128, 0.024609722, 0.0077196537, 0.0074068746, 0.0059062005, 0.019139417, -0.018593717, -0.016717043, -0.006491829, -0.016091485, -0.0038132425, -0.0001429758, 0.023917615, 0.04895325, 0.032289445, 0.008524893, -0.037027717, -0.014174881, -0.012624295, -0.015000085, -0.027497938, 0.014960156, -0.014827059, 0.0025438254, -0.025687812, 0.028855532, 0.03532407, -0.036122654, -0.024929157, 0.021721508, -0.019392302, -0.041473173, 0.015599024, -0.045492716, -0.048740294, -0.013948616, 0.0030329586, 0.02532845, -0.025142113, 0.0006001864, -0.015692191, 0.03215635, -0.013362986, 0.02519535, -0.00028574365, -0.018181115, -0.049938172, -0.014081713, -0.0326355, -0.046158202, -0.060958643, 0.021322215, -0.04482723, -0.017821752, 0.019392302, -0.027764132, 0.0055967486, -0.034951396, 0.013815518, 0.011912224, -0.0031028348, -0.00065134576, -0.03841193, -0.030239746, -0.049086347, 0.034924775, -0.057870783, 0.0037766409, 0.04799495, 0.020803135, -0.011220117, 0.052387167, 0.016929999, -0.004358942, 0.025954008, -0.015399378, 0.022985933, 0.0031494189, -0.014041783, -0.050816614, 0.01343619, -0.024024094, 0.012604331, -0.008870946, 0.013456155, -0.0163843, -0.022213968, -0.03881122, 0.025514785, 0.015412687, 0.00815222, -0.029281445, 0.01372235, -0.008431725, 0.007945919, 0.055049114, -0.051295765, -0.011932189, -0.026446467, -0.037506867, -0.019991241, 0.050710138, -0.005620041, -0.021268977, 0.011180188, 0.0023674713, -0.023890996, 0.02423705, 0.019086178, 0.019179346, -0.019565329, -0.017595487, 0.0067912987, -0.049565498, -0.029840454, 0.02377121, -0.01473389, 0.017608795, -0.04935254, -0.003966305, 0.01744908, 0.0031976667, 0.006794626, 0.0022227278, 0.0067779887, 0.03585646, -0.0007349476, -0.020430462, 0.00068753166, -0.027338222, -0.011792436, -0.0034023041, -0.008418415, 0.063940026, -0.000114276656, -0.0274447, -0.028403, -0.012544437, -0.00819215, 0.018580407, -0.012770702, -0.005157527, -0.015998317, 0.039290376, 0.023079101, -0.007327016, 0.0060060234, -0.0060758996, -0.009909107, -0.011918879, -0.0064485725, 0.0035137732, -0.014920227, 0.0015081607, -0.004678376, 0.00047998276, -0.026060486, 0.019631878, -0.042378236, 0.011639374, 0.030479321, 0.0015705501, -0.040727828, -0.014827059, 0.027577797, 0.007200573, 0.0010348329, 0.03894432, 0.022520093, 0.0007120715, -0.02792385, -0.018873222, -0.0006808768, 0.007327016, -0.011912224, 0.005796395, 0.00845169, 0.004096075, -0.039769527, 0.0113332495, 0.024370147, -0.003061242, 0.009250275, -0.004974518, -0.015878528, -0.048287764, -0.019565329, 0.011313286, 0.041260216, 0.010215231, -0.014427766, -0.03484492, 0.033327606, -0.01343619, -0.0071539893, 0.011133604, 0.038225595, -0.010840789, -0.006398661, 0.003936358, -0.013935305, -0.029281445, -0.007440149, 0.003793278, 0.03433915, -0.010987196, -0.034578722, -0.014494315, 0.039583188, 0.03159734, 0.021215737, -0.07267122, 0.058136977, 0.018247664, 0.019578638, 0.028882151, -0.0018483911, -0.011040435, 0.025208661, -0.03423267, 0.011639374, 0.0050377394, -0.0273116, -0.046131585, -0.02104271, 4.406254e-05, -0.034072954, -0.0037999328, -0.006764679, -0.022213968, 0.03111819, 0.030053409, 0.027524557, -0.10658446, -0.040461633, 0.012378065, -0.022480164, 0.015266281, 0.009849213, 0.016104795, -0.014148261, 0.015266281, -0.046451017, 0.03159734, -0.04903311, 0.03103833, -0.006228962, -0.006604962, -0.06575015, -0.009030663, 0.022240588, 0.07756921, -0.015146492, -0.019339062, 0.015346139, 0.016464157, 0.018993009, -0.009802629, -0.020789826, -0.002866587, 0.039849382, 0.048261143, 0.001424143, -0.015465926, 0.019685116, 0.00084267335, 0.042777527, -0.019778285, 0.00041218623, 0.003267543, -0.024835989, -0.0057830852, -0.025594644, -0.0074002193, -0.032236207, 0.00895746, -0.003403968, -0.0008193813, 0.013988544, 0.0028532771, 0.016118104, -0.023345297, 0.0017219486, 0.0025920733, 0.012105215, -0.004305703, 0.035057876, 0.01840738, -0.014041783, -0.0084649995, -0.0021644977, -0.0018600372, 0.031357765, -0.004358942, 0.019392302, -0.027790753, -0.008098981, -0.032715358, -0.011938843, 0.019032938, 0.016171344, -0.0013725677, -0.023371916, -0.034152813, 0.034685202, 0.0133496765, 0.016118104, 0.0066748383, -0.010927303, 0.022812907, 0.059734147, -0.0128838355, -0.017728584, 0.027471319, -0.054942638, -0.0022776306, 0.0055934214, 0.014028474, 0.036468707, 0.03532407, -0.03367366, -0.010155337, 0.016224582, -0.0031161446, -0.0008692929, 0.020696657, 0.014560863, -0.008751159, 0.013456155, -0.0055867666, 0.037267294, 0.014614102, -0.0006492661, 0.040435012, -0.007327016, -0.018859912, -0.04304372, 0.014920227, -0.014520935, 0.009596328, -0.0022593297, -3.3118395e-05, 0.022293827, 0.023984164, 0.0389177, 0.006175723, 0.044401318, -0.008691265, 0.021362145, 0.05638009, 0.02164165, 0.04831438, 0.010308399, 0.022573331, -0.013103446, -0.0041992255, -0.009669531, 0.010614524, -0.0124446135, 0.029680736, 0.039104037, 0.02371797, -0.0012261604, -0.025967317, 0.032342687, -0.0221208, -0.009310168, -0.035563644, -0.020776516, 0.014467696, -0.029387923, -0.01130663, 0.014321288, -0.01686345, -0.0039396854, 0.051881395, -0.017129645, 0.006122484, 0.004229172, -0.028376382, -0.031677198, -0.008691265, -0.0061125015, 0.032049872, -0.0076198303, -0.01736922, 0.050683517, -0.019605258, 0.020922923, -0.0008405937, -0.058243454, -0.006385351, 0.018008089, 0.0050077923, 0.0077129984, 0.00475158, -0.007899335, 0.02896201, 0.013389606, -0.027897231, 0.016703732, -0.01131994, 0.009083902, 0.0084649995, -0.0063953334, 0.040914163, 0.03889108, 0.021721508, 0.018607026, 0.0021644977, -0.011532896, 0.018141186, 0.013908686, -0.030985093, 0.041473173, -0.00841176, 0.039902624, 0.017568866, 0.0019116125, 0.015519165, -0.008657991, -0.0010489745, 0.01848724, 0.003142764, 0.026153654, -0.006032643, 0.0014773819, 0.014654032, -0.012544437, 0.056539807, 0.020177577, -0.018660266, -0.022426924, 0.004172606, 0.07229855, -0.006618272, 0.0155724045, -0.0348183, -0.005383793, -0.021362145, -0.01424143, 0.015532475, 0.0056932443, -0.013895377, 0.0046484293, -0.010328364, 0.0043123583, -0.0023175597, -0.011925533, 0.005902873, -0.00554351, -0.010880718, -0.035164353, -0.045332998, -0.016796902, -0.040168818, 0.034392387, -0.007420184, -0.011652684, -0.027085336, -0.013908686, -0.028163426, -0.0011063728, -0.010461462, 8.344588e-05, -0.007799512, 0.0027867283, -0.04067459, 0.0095763635, 0.0009998948, -0.015040015, -0.007067476, 0.014454385, 0.03684138, 0.0137888985, -0.025009016, 0.002061347, 0.0040328535, -0.019565329, 0.008558167, 0.0015522493, -0.019685116, -0.030718897, 0.009070593, 0.0122649325, -0.0012136826, -0.017835062, 0.0050943056, -0.010794205, 0.03625575, 0.052120972, 0.02952102, 0.005606731, -0.0133496765, 0.056007415, 0.0137888985, -0.0015089926, 0.0056100585, -0.0003857747, 0.0006434431, 0.014640722, 0.03008003, 0.0024489935, -0.023052482, 0.034152813, 2.0406545e-05, 0.025488166, -0.0035470477, 0.043815687, 0.028509479, -0.00869792, 0.011632719, 0.0021961082, -0.012750738, -0.022014322, -0.011932189, 0.0022959313, -0.009250275, 0.020350603, -0.016477467, -0.0021478606, 0.06968984, -0.03218297, 0.0009566381, 0.05017775, 0.02113588, 0.0071273698, -0.009629603, 0.027977088, -0.013209924, -0.026300061, 0.037986018, -0.056060657, 0.034498867, 0.0019166037, 0.018074637, -0.041978944, -0.005380465, 0.016703732, 0.021814676, -0.02579429, 0.01317665, 0.0095763635, -0.016264511, 0.0026752593, 0.004934589, -0.012431304, 0.01630444, 0.011166878, -0.0111269485, -0.0071406793, -0.009942382, -0.034152813, -0.010594559, 0.0151331825, -0.0004874695, -0.02842962, 0.018154494, 0.0069676526, 0.012351446, -0.008664645, -0.008145565, -0.006748042, 0.029574258, -0.00023271261, 0.0002865755, 0.023957545, -0.040222056, -0.02890877, -0.032289445, -0.030639037, 0.014174881, -0.033088032, 0.014933537, 0.016637184, -0.0063886787, -0.015412687, 0.011639374, 0.010607868, 0.009449921, -0.03513773, 0.044135123, -0.013522703, 0.061331317, -0.0051109428, -0.011260047, -0.019991241, -0.016091485, 0.043576114, -0.007846096, 0.007819477, -0.040781066, -0.023331987, 0.020776516, 0.022320446, -0.0073935646, 0.0036368885, 0.026579566, 0.02004448, -0.025860839, 0.016717043, 0.023039173, 0.010168647, 0.005773103, -0.0016753644, -0.005317244, 0.0075066974, 0.012258277, 0.015585714, -0.0006276378, 0.031171428, -0.012451269, 0.009902452, -0.02057687, -0.025701122, 0.010308399, 0.014520935, -0.019764975, 0.0146673415, 0.008890911, 0.020217506, -0.0012452932, -0.017582176, 0.017249433, 0.0007653105, -0.025674503, 0.0034771715, -0.014853678, -0.004235827, 0.0082320785, 0.020670038, -0.019352373, 0.007493388, -0.011113639, -0.0325024, 0.0038498444, 0.012038667, 0.04059473, -0.021455312, 0.004285739, 0.04743594, -0.010581249, 0.020430462, -0.030479321, 0.01686345, -0.0473827, 0.025368378, 0.0029730648, -0.04991155, -0.00872454, 0.031490862, -0.043017104, 0.028456239, -0.032529023, -0.0049179518, 0.01342288, 0.0040561454, 0.010261815, 0.013709039, -0.0011354878, -0.0057065543, 0.024969086, 0.052493643, 0.010128718, 0.015053324, -0.009596328, 0.0046850313, 0.013828827, -0.0059294924, 0.026739283, -0.01473389, -0.025927387, 0.047648896, 0.0020580196, 0.023877686, 0.019245895, 0.04887339, -0.005566802, 0.01896639, -0.013516049, 0.0062655634, 0.027631035, 0.0063154753, 0.0142547395, -0.005603404, -0.031304527, 0.03785292, -0.007686379, 0.012484543, -0.008504929, -0.009729425, 0.0018750107, 0.0027201795, 0.013855447, -0.00068919535, 0.0069809626, -0.00947654, 0.019565329, 0.011140259, 0.006748042, 0.03684138, -0.0060925367, -0.02994693, 0.028163426, 0.016557327, 0.034472246, 0.027604416, 0.01581198, -0.009270239, -0.0041992255, -0.015638953, -0.03952995, -0.009423302, -0.030239746, 0.021335525, 0.016011626, -0.030239746, 0.020350603, 0.009256929, -0.009642912, -0.004791509, -0.028749054, 0.010401567, 0.040435012, 0.03476506, -0.0080257775, -0.0011770808, -0.00841176, -0.020230817, 0.012251622, 0.011013816, -0.0010747621, -0.022746358, -0.0049645356, -0.033460706, 0.00619236, 0.014547554, 0.0043156855, -0.012823941, 0.0032459146, -0.0063920063, 0.0082320785, -0.007939264, -0.0076131755, -0.012145145, -0.02787061, 0.009676186, 0.022586642, -0.0113598695, -0.0028266576, -0.0006575847, -0.010401567, -0.035483785, 0.011033781, 0.015905147, 0.0035570299, 0.001790161, -0.017249433, -0.0076730694, -0.03306141, 0.0045186593, 0.004485385, 0.017276052, 0.010348328, 0.012584366, -0.011193497, -0.024064023, 0.00088925747, -0.028615957, 0.006628254, 0.00593282, -0.0086713005, 0.0033174546, 0.03367366, -0.023465084, -0.026193583, -0.011067055, 0.0033074722, 0.053318847, 0.0142015, 0.038092498, 0.0558477, -0.007546627, 0.011699268, 0.0075200074, -0.02524859, 0.00447873, -0.02584753, 0.017715273, 0.011785781, 0.020882994, 0.0089308405, 0.018074637, 0.0013858775, 0.013569287, -0.0057265186, 0.0016429219, 0.03465858, -0.015026704, 0.0008850982, -0.009063938, 0.0026153654, -0.020776516, 0.019565329, -0.009443265, 0.0305858, 0.028509479, 0.0044986946, -0.013030243, 0.011040435, -0.007200573, 0.009250275, 0.008225424, 0.003626906, -0.020789826, 0.0005623368, 0.009929071, 0.011033781, 0.007839441, -0.020643419, -0.011100329, -0.0061491034, 0.011958808, 0.029707355, 0.029707355, 0.015186422, 0.01586522, -0.020124339, 0.027897231, 0.031171428, -0.0084383795, -0.038252212, -0.0058762534, -0.011532896, 0.026060486, -0.027710894, -0.031517483, -0.0043289955, -0.019898072, -0.008079017, 0.024396766, 0.031783678, 0.053372085, -0.005713209, -0.0038664816, 0.008731194, 0.024064023, -0.002429029, 0.053478565, -0.014534244, -0.0129370745, -0.028083567, 0.01846062, -0.029414542, -0.010115408, 0.014055093, -0.0104814265, -0.040701207, 0.026127035, -0.013083481, -0.0049445713, -0.011240082, 0.0043323226, -0.010448151, 0.008611406, 0.0030762153, -0.0097560445, 0.0045519336, 0.029787214, 0.024356838, -0.004731615, 0.0036135963, 0.01187895, -0.0073003964, 0.012417994, 0.00924362, -0.02158841, 0.01577205, 0.03215635, -0.016464157, -0.030186508, -0.029867074, 0.026592875, -0.0073602903, -0.009063938, -0.014334598, 0.02113588, 0.031490862, 0.014427766, -0.014307979, 0.036548566, 0.011200152, 0.014121642, 0.010993851, -0.024662962, -0.04152641, 0.0029580914, 0.019591948, -0.004927934, -0.017009858, 0.01080086, 0.0032059853, 0.05177492, 0.0034272599, 0.010927303, 0.023944236, -0.006252254, 0.024822678, -0.01026847, -0.00070167327, 0.031490862, 0.010721002, -0.014108332, -0.008598097, -0.017475698, 0.025741052, -0.0016104794, -0.020310674, 0.032715358, -0.016969929, 0.020749897, 0.06250257, -0.00841176, -0.040355153, 0.015758742, -0.05326561, -0.025514785, 0.005174164, 0.046530876, -0.00738691, -0.030346224, 0.002151188, 0.004804819, 0.058882322, 0.048048187, -0.004508677, 0.0055202176, 0.028056948, 0.0061890325, -0.00738691, -0.0053205714, -0.015239661, -0.021295596, -0.0503907, 0.0071140598, 0.021827986, 0.01689007, -0.013116756, 0.0029647462, 0.0060925367, -0.037240673, -0.0077063437, 0.0046850313, -0.0017419131, 0.040887542, 0.025927387, 0.00012134746, 9.2908274e-05, 0.016410919, 0.009596328, 0.0012403021, 0.012298207, -0.014933537, 0.0137622785, -0.004285739, 0.014481005, -0.013343021, 0.005254023, -0.021415383, 0.008524893, 0.02164165, 0.0042691017], "content_hash": "8fd31ca09bd50da94178847772236eba"}, {"content": "# Introduction to Mechanistic Interpretability\n\n## What Is Mechanistic Interpretability?\n\nMechanistic interpretability (often called \"mech interp\") is a field of AI research that aims to understand **how** neural networks work internally -- not just what they predict, but why. Instead of treating models as black boxes, researchers open them up and study the individual components (neurons, attention heads, layers) to figure out what each one does.\n\nThis dashboard is a tool for doing exactly that kind of investigation.\n\n## How This Dashboard Relates\n\nThe experiments available in this dashboard are real techniques used in mechanistic interpretability research:\n\n- **Ablation** (removing heads to test their importance) is a standard tool for identifying which components are responsible for specific behaviors\n- **Token attribution** (measuring input influence via gradients) is used to trace how information flows from inputs to outputs\n- **Attention pattern analysis** (categorizing heads by behavior) helps researchers build a map of what each head does\n- **Head categorization** (Previous-Token, BoW, Syntactic, etc.) builds on research that has identified recurring head types across models\n\n## Key Concepts in the Field\n\n### Circuits\n\nA **circuit** is a small subnetwork within the model that performs a specific function. For example, researchers have found \"induction circuits\" -- combinations of attention heads across layers that work together to complete patterns like \"A B ... A\" \u2192 \"B\" (if the model has seen \"A B\" before, when it sees \"A\" again, it predicts \"B\").\n\nIn the dashboard, you can start to identify circuits by ablating combinations of heads and seeing which combinations have outsized effects.\n\n### Superposition", "source_file": "mechanistic_interpretability_intro.md", "chunk_index": 0, "embedding": [-0.026108723, 0.036923856, -0.007753743, 0.01745929, -0.002281004, -0.01040071, 0.0033270908, 0.0187293, -0.012071775, 0.024451027, 0.0079475865, -0.05093407, -0.022860173, -0.011162716, 0.007981008, -0.035399847, -0.008435537, 0.010126655, 0.015494117, 0.04858121, 0.003930345, -0.024437658, 0.044436965, 0.060372245, 0.009906075, -0.007332634, 0.004765878, 0.03320741, 0.008602643, 0.019638358, 0.020948473, -0.012673358, -0.028795796, -0.028073896, -0.0245045, -0.020146362, 0.008743013, -0.012045038, -0.012479515, 0.029517695, 0.011303085, -0.015881805, -0.029731592, 0.03852808, -0.05598737, 0.007887428, -0.0024932292, -0.023541966, -0.017686555, 0.056468636, -0.046816565, 0.05047954, -0.020092888, -0.0016134134, 0.013756209, 0.040987886, -0.046789825, -0.0052905926, -0.01071487, -0.020386996, -0.018956564, -0.0061896257, 0.03280635, 0.033849098, -0.037217963, -0.019691832, -0.013455417, 0.020681104, -0.0003511326, -0.0021991218, 0.057965912, 0.023167647, -0.0118913, 0.021670373, 0.0017796844, -0.060265295, -0.036843646, 0.03842113, -0.021870902, -0.0149727445, -0.0064503117, -0.009137385, 0.01979878, -0.055933896, -0.039784722, -0.026402831, -0.027004413, 0.006924894, -0.02260617, -0.06973021, -0.01076166, 0.017405815, -0.018167822, 0.011837826, 0.028795796, 0.0069516315, -0.030961497, -0.011503613, -0.005561305, 0.036335643, 0.0077002686, -0.04216432, 0.008930173, -0.0048260363, 0.044169597, -0.052458078, 0.0018532113, -0.025507139, -0.028501688, -0.036041535, -0.03534637, -0.010888661, -0.013361838, 0.0034039598, 0.0037431861, -0.010206866, -0.041362207, -0.012065091, -0.013575734, -0.029811803, -0.029651381, -0.01863572, 0.032030977, 0.0036529487, -0.008415485, -0.05162923, 0.019544778, -0.007867375, -0.01626949, -0.006099388, 0.002794021, 0.038127024, -0.02044047, -0.07138791, -0.0051101176, -0.033501517, 0.006680919, 0.034784894, -0.051147964, -0.0114969285, 0.011503613, -0.003766581, 0.01003976, 0.007987692, -0.057217274, 0.007693584, 0.035292897, 0.05063996, -0.01990573, 0.033875834, -0.0028508373, 0.0020704498, 0.014758848, -0.07299213, -0.023956392, -0.011463507, -0.0038233972, 0.043848753, 0.015400537, -0.021376267, 0.05473073, 0.028207581, -0.013368522, -0.0072657918, -0.06277858, 0.02767284, -0.029918753, 0.01620265, -0.023194386, -0.02098858, -0.025092715, -0.013983474, -0.017165182, -0.024491131, 0.025801247, -0.013014256, -0.06352722, -0.03229835, 0.028501688, -0.018194558, -0.0077938484, -0.0054610413, -0.035319634, 0.014464741, -0.027993685, -0.012145302, -0.042378213, 0.030506967, -0.014825691, -0.030105911, -0.022312064, 0.06261816, 0.02007952, 0.012225513, -0.0068079196, -0.037378386, 0.052163973, 0.007018474, 0.063901536, -0.033581726, 0.002257609, 0.015106429, 0.0672704, 0.008368694, 0.029597906, 0.063206375, 0.019237302, 0.015667908, -0.021028684, -0.036121745, 0.009431493, -0.022485854, 0.03470468, 0.016937917, -0.06539881, -0.039303456, -0.04130873, 0.046415508, 0.035025526, 0.039677773, -0.005942308, 0.007954271, 0.025253138, 0.017432552, -0.014999482, 0.037832916, 0.015093061, -0.014812322, 0.016109068, -0.012693412, 0.018782774, 0.0028023764, -0.0082416935, 0.022499222, 0.0038802135, -0.055826947, 0.013889894, -0.024651553, 0.040667042, 0.0033204067, 0.025279874, 0.0120116165, -0.003927003, 0.019411094, 0.029624645, 0.021456478, -0.018836247, -0.01600212, -0.017138446, 0.034945317, 0.049089212, 0.047618676, -0.0656127, 0.02116237, -0.021991218, 0.0038534764, -0.026723675, 0.0017345657, -0.013903263, 0.039089557, -0.023555335, -0.02332807, -0.05588042, -0.005668253, -0.02586809, -0.04141568, -0.0052103815, -0.0036061588, -0.058821496, -0.027458943, 0.0059556765, 0.023488494, -0.002461479, 0.026710307, -0.07336645, -0.045613397, 0.015253483, 0.0245045, -0.016563598, 0.0016627099, -0.03609501, -0.01953141, 0.089248255, 0.0108285025, -0.05978403, 0.011657351, 0.0014237476, -0.020320153, 0.009451545, -0.028261054, -0.020935105, 8.71565e-05, 0.025279874, 0.0016033871, -0.019424463, -0.017419184, 0.01239262, 0.019558147, 0.033474777, 0.011724194, 0.051522285, -0.056361686, 0.010841872, 0.008027798, 0.027432207, 0.014464741, 0.0012775294, 0.0014112146, 0.045747083, -0.0020587523, -0.03128234, -0.042084105, 0.019304145, 0.0066374713, -0.0230607, 0.0145315835, 0.0468433, 0.0018766063, 0.035827637, -0.048153415, -0.018715931, -0.015333694, 0.027646104, -0.009331228, -0.0014630177, 0.020627629, -0.0017613027, 0.009217596, -0.037592284, 0.0004787602, 0.056415163, -0.018288137, 0.002590151, 0.017071603, -0.025761142, -0.009478282, -0.027178206, 0.0062330733, -0.05109449, 0.042351477, -0.04585403, -0.004588745, -0.07085317, 0.035399847, 0.033394568, 0.040827464, -0.035747427, -0.017312236, 0.04312685, 0.005410909, -0.018675825, -0.024263866, -0.029544434, -0.06267163, -0.021763954, 0.062190365, -0.04256537, -0.013174678, 0.022646276, -0.019638358, -0.039036084, 0.014077053, 0.004739141, 0.027619366, -0.01402358, -0.023849443, -0.038447868, -0.0006258139, -0.016068963, 0.0013452074, 0.02433071, -0.010708186, -0.014397898, -0.010073181, -0.04882184, 0.00087062496, -0.028020421, -0.019638358, -0.03636238, 0.03192403, 0.046522457, 0.0050499593, -0.018515402, -0.032378558, 0.025547246, 0.026603358, 0.014705374, -0.029785067, -0.06427585, 0.036522802, 0.020747947, -0.05518526, 0.03165666, 0.022084799, -0.029704856, 0.019571517, -0.0052371183, 0.0019284092, 0.049891323, 0.019691832, -0.0056348317, 0.006019177, -0.0600514, -0.018929826, 0.029785067, 0.047458254, 0.013742841, 0.021763954, -0.016536862, 0.030105911, 0.056575585, -0.026603358, 0.027485682, 0.011664035, 0.013769577, -0.03235182, 0.023354808, 0.0084088, 0.035961322, 0.02171048, -0.0149727445, -0.027913474, 0.005013196, -0.010093234, 0.00053891854, 0.05243134, 0.043715067, -0.024156919, 0.043153588, -0.076200575, -0.000709785, 0.045105394, 0.04772562, -0.024156919, 0.0029360617, -0.026857361, 0.067003034, -0.001487248, -0.01270678, -0.008482327, 0.037217963, -0.016790863, 0.0031198787, -0.011403349, -0.042725794, 0.0040506623, -0.021723848, -0.09138721, 0.007245739, -0.06058614, 0.02388955, 0.013321732, 0.03454426, 0.01402358, -0.016884442, -0.035774164, 0.06384806, 0.002075463, 0.009244333, -0.006690945, 0.041442417, 0.022766592, -0.017766766, 0.05887497, 0.001565788, 0.022084799, -0.03716449, -0.041041363, 0.006530523, -0.041362207, 0.011623929, -0.027164835, 0.03355499, 0.0060659666, -0.033768885, -0.03572069, 0.057110325, -0.020788051, 0.0048795105, -0.027258417, -0.0073058973, -0.0032819721, -0.013729472, -0.014224107, 0.03751207, 0.050666697, -0.05534568, -0.03606827, 0.035587005, 0.019050144, -0.025119452, -0.024170287, 0.0042077424, -0.014718742, 0.009444861, -0.011062452, -0.047591936, -0.044089384, -0.0012023314, 0.02088163, 0.03427689, -0.015253483, -0.011837826, 0.057270747, 0.00501988, 0.038394395, 0.010988925, -0.0050098537, -0.030801075, 0.013074414, -0.011336506, -0.0293038, -0.0035159213, 0.029277062, -0.0097256, 0.011710825, 0.018702563, 0.008529117, 0.052057024, -0.006346706, -0.013375206, -0.028154107, 0.0025734405, -0.0035226054, 0.00450185, -0.008235009, -0.05614779, -0.036977332, -0.054677255, -0.0077470583, 0.0030363256, -0.01383642, 0.014865796, 0.029785067, -0.03040002, -0.014732111, 0.013943369, -0.026523147, -0.03924998, 0.02804716, 0.032432035, 0.014411267, 0.0055245417, 0.01700476, -0.017793503, 0.012446093, -0.006784525, -0.040667042, 0.024357447, 0.039731245, 0.0059022023, -0.037859652, 0.03981146, -0.03761902, -0.015587697, -0.007025158, 0.0020637657, -0.02577451, 0.025440296, -0.014250844, -0.021630269, -0.028956218, -0.006126125, -0.012840466, -0.008950225, 0.0051635914, 0.010200182, -0.024651553, 0.025012504, -0.02133616, -0.0024297289, 0.0011112584, -0.00018809928, 0.010013022, 0.031362552, -0.007994376, -0.016750759, -0.007813901, -0.0013385232, -0.0020620944, -0.030239597, -0.02268638, -0.034972053, 0.024290605, -0.02695094, 0.032512244, 0.017579606, -0.007813901, -0.018943196, -0.008295168, -0.014798953, -0.019143723, 0.0060659666, 0.033341095, -0.022820067, 0.0068246303, 0.016991392, 0.01763308, -0.034303628, 0.031362552, -0.003126563, -0.0049263, 0.002921022, -0.009939496, -0.01520001, 0.0083152205, -0.033501517, 0.032726143, 0.010587869, -0.013134573, -0.01998594, 0.0056214635, 0.0025700983, -0.022111535, -0.062083416, 0.05363451, -0.025654193, 0.013782946, 0.011503613, -0.016135806, -0.00542762, -0.013254889, 0.0008372037, -0.032137927, 0.023100806, 0.0023729126, 0.010607922, -0.02334144, 0.01402358, -0.005527884, -0.025827983, 0.008856646, 0.035480056, -0.0013000887, 0.007192265, 0.007767111, 0.013809683, -0.0036763435, 0.047324568, -0.0111292945, 0.010253657, -0.033849098, -0.003046352, 0.00087730924, 0.019691832, -0.014705374, 0.010654712, 0.025426928, -0.0343571, -0.005688306, -0.039276715, -0.023301333, 0.0468433, 0.012680043, 0.0036596328, -0.005938966, 0.011289717, -0.015213378, -0.022231853, 0.023127543, -0.0032719458, 0.009525072, -0.004912932, -0.031095183, -0.025199663, 0.020119626, 0.02296712, 0.012653306, 0.010013022, 0.025988406, -0.041736525, 0.017071603, 0.0098526, 0.026630096, 0.01717855, 0.0035493425, 0.013675998, -0.007018474, -0.036415853, -0.0016576967, 0.023127543, 0.01184451, -0.012225513, 0.013595787, 0.00041317087, 0.0081681665, 0.038314182, 0.004745825, 0.001626782, -0.03887566, 0.010480921, -0.0040473198, 0.010440815, -0.0045419554, -0.032378558, 0.0048126676, 0.023435019, 0.019237302, -0.0062063364, -0.10293762, -0.0003172935, -0.026603358, -0.003455763, -0.008522432, -0.003636238, -0.0098526, 0.042538635, 0.033474777, 0.018421823, -0.06186952, -0.07021148, 0.012994203, 0.00814143, 0.025827983, -0.0040840833, 0.029063167, -0.010053128, 0.0020270022, 0.006019177, -0.032886565, -0.051602494, 0.020026045, -0.05769854, -0.002177398, 0.0151197985, -0.016857706, -0.022753224, 0.0075398465, -0.017125076, 0.009645388, 0.004531929, 0.014103791, -0.011851194, -0.0687142, 0.004896221, -0.0015031232, -0.0010335539, -0.006343364, -0.009077227, -0.007346003, -0.0058754655, 0.016630441, 0.0031416027, -0.00063918246, -0.019611621, 0.03545332, -0.017405815, 0.00886333, -0.024357447, -0.04096115, -0.008087955, 0.011764299, -0.015173272, 0.015962016, 0.0012842136, 0.021389635, 0.012218829, -0.041816738, 0.01565454, 0.0068981573, -0.026309252, 0.020386996, -0.008495696, 0.013228152, -0.03465121, -0.036041535, 0.058714546, -0.012593147, 0.011998248, 0.03590785, 0.020293416, -0.007974323, 0.024076708, -0.013609155, 0.006152862, -0.024918925, 0.042191055, 0.014224107, 0.013355154, 0.0119380895, -0.03184382, -0.014518214, -0.021763954, -0.01637644, 0.00046706272, 0.03040002, -0.053206716, -0.032030977, -0.021028684, -0.033982784, -0.04205737, 0.0025416901, 0.010567817, -0.03427689, -0.014758848, -0.0025467034, -0.0015081364, -0.08823224, -0.009117332, 0.025600718, -0.010327183, 0.0044517177, -0.0045753764, 0.029143378, 0.016750759, 0.017218657, -0.0656127, -0.0023712416, -0.0007666012, 0.021242581, 0.019103618, 0.028100632, -0.028688848, -0.008174852, 0.0013794643, 0.037832916, 0.026309252, -0.01979878, 0.019838886, -0.00773369, -0.008308536, -0.057270747, -0.020132994, -0.019772043, 0.006323311, 0.026710307, 0.018702563, -0.030667389, 0.002974496, -0.041843474, 0.006607392, -0.005608095, 0.020574154, 0.038180497, 0.008248378, 0.015293589, 0.0067945514, -0.0053073033, -0.011470192, 0.014210738, -0.005430962, 0.04676309, 0.021175738, 0.021175738, -0.012506252, -0.01691118, 0.03879545, -0.0072858445, 0.003666317, 0.025841353, -0.0063634166, 0.021937745, -0.0055847, 0.0013443719, 0.026710307, 0.022566065, -0.00037494526, 0.003970451, 0.008829908, 0.032378558, 0.014932639, -0.00097506656, 0.038127024, 0.007693584, 0.012940729, -0.030613914, -0.007833954, -0.0021189107, -0.004404928, -0.020453839, 0.031496238, 0.0101667605, -0.02171048, 0.020226574, -0.005735096, -0.0007845651, -0.015547591, 0.04593424, -0.012325777, -0.039330192, 0.025747772, -0.005123486, 0.020280048, -0.014117159, 0.011283033, -0.0014897546, 0.034731418, 0.019892361, 0.01943783, 0.012780307, -0.019116987, -0.0043180324, 0.033608463, 0.0022826751, 0.037191227, 0.09625336, -0.004558666, 0.020333521, 0.027512418, -0.0011521995, -0.012593147, -0.03644259, 0.022900278, -0.038768712, -0.014143896, 0.01157714, 0.017486027, -0.011750931, 0.020814788, -0.022472486, -0.007847322, 0.013415311, 0.0112028215, 0.017191919, 0.0027605998, 0.0069783684, -0.018675825, 0.010253657, -0.01620265, -0.03425015, -0.0041175047, 0.01763308, 0.030640652, 0.035426583, 0.0066274446, 0.017325604, 0.010995609, -0.021804059, 0.0064904173, 0.01438453, -0.0018415139, -0.016429914, -0.005922255, 0.003666317, 0.013421996, -0.008181536, 0.029597906, -0.042806007, -0.011162716, -0.0008388747, 0.012131933, -0.012145302, 0.007579952, -0.03138929, -0.034062993, -0.026723675, 0.020600893, 0.0020370285, 0.035132475, 0.006336679, 0.037993338, -0.009638704, 0.05652211, 0.010146708, -0.05903539, -0.028073896, 0.031416025, -0.007352687, 0.027539155, 0.010741607, 0.007332634, 0.0111292945, 0.019103618, -0.02205806, 0.008743013, -0.009197543, 0.035961322, -0.047859307, -0.008482327, 0.010153392, 0.010875292, -0.0023261227, 0.011617245, -0.0035927903, 0.018100979, 0.014103791, 0.0056348317, -0.0048795105, 0.039677773, -0.016470019, 0.008702908, 0.019865623, -0.023287965, 0.008803172, -0.023394912, -0.032325085, -0.009484966, 0.017338973, 0.004759194, -0.0057985964, 8.5642096e-05, -0.039490614, -0.011978196, -0.006537207, 0.032244876, -0.014678637, 0.005437646, -0.05652211, 0.0180074, 0.005688306, -0.032726143, 0.010246972, 0.028341265, 0.032619193, -0.0036562907, -0.010521026, -0.021042053, -0.024918925, -0.018047504, 0.0066374713, 0.034758158, 0.014237476, 0.046362035, 0.01645665, -0.009050489, 0.0114969285, -0.0223388, -0.014237476, -0.041469153, -0.021723848, 0.014905902, -0.008943541, 0.031335816, -0.02414355, 0.0019835543, 0.020640997, 0.002408005, -0.0034992106, 0.0038133708, -0.0036830278, 0.004020583, 0.0051301704, 0.015935278, 0.008355326, 0.009484966, 0.05197681, -0.018167822, 0.014986113, 0.003505895, -0.0343571, -0.02658999, -0.016710652, -0.030560441, 0.03860829, 0.025467034, 0.0057284115, -0.013488838, -0.0032000898, 0.033929307, 0.04793952, -0.021550057, 0.013448733, -0.028608637, 0.04721762, 0.030801075, 0.017071603, 0.002742218, 0.0372447, 0.01808761, 0.023996497, -0.019477936, 0.016603705, -0.013889894, -0.0038868978, 0.00041922848, -0.013903263, 0.024584712, -0.0020871605, 0.025266506, 0.011423402, -0.0052103815, -0.022312064, 0.00011008142, -0.030961497, -0.007660163, -0.0128270965, -0.02097521, -0.0017980662, 0.008208272, 0.017138446, 0.013575734, 0.014986113, -0.0058520706, -0.031335816, -0.002382939, 0.010581185, -0.029865278, -0.030827811, 0.011135979, 0.028715584, -0.042645585, -0.03625543, 0.007118738, -0.030720863, -0.013756209, -0.00800106, -0.022779962, 0.008883383, 0.013595787, -0.014825691, -0.01807424, 0.025573982, -0.038661767, 0.020921737, -0.008582591, -0.0048093256, 0.021149002, -0.03491858, -0.0024013207, 0.020106256, 0.03280635, 0.029330537, -0.0023010569, 0.015360432, -0.047083933, 0.00030183615, -0.01592191, -0.003793318, 0.032432035, 0.0004449211, 0.020132994, -0.013669314, -0.0009642046, -0.0016468348, -0.0007402819, 0.0019000012, -0.017338973, -0.018328244, 0.017379079, -0.010995609, -0.0074863723, -0.013288311, 0.027966948, -0.005922255, -0.047458254, 0.015534222, 0.014718742, -0.00872296, 0.019571517, 0.01717855, -0.0063032582, 0.0073192655, -0.010053128, 0.0043080063, -0.0189833, 0.02759263, 0.031041708, 0.06427585, -0.0035961324, -0.0058888337, 0.008749697, 0.0027004415, 0.0041208467, -0.033153933, 0.009077227, -0.046415508, -0.013903263, -0.01438453, 0.053554296, 0.015962016, -0.0016334662, -0.0066207605, -0.0012457791, -0.065452285, -0.0016008805, -0.0063868114, 0.029063167, 0.00517696, 0.0034758158, -0.016657177, -0.015320326, 0.028982956, -0.011389981, 0.022017956, 0.02577451, 0.027178206, 0.018929826, -0.0025801247, -0.025721036, 0.026523147, 0.0022191745, 0.0064937593, 0.009638704, 0.018047504, 0.0010135011, -0.039544087, -0.011924721, 0.035827637, 0.0055445945, 0.0031031682, -0.026803887, 0.0014939323, 0.026683569, -0.028368004, -0.003459105, -0.021322792, -0.014919271, -0.03328762, -0.006767814, 0.014665268, -0.008990331, 0.0034106441, -0.014491478, 0.0025433612, 0.047645412, -0.0073192655, 0.014665268, -0.0099461805, 0.03355499, -0.013034308, -0.0057785437, 0.012639937, -0.023822706, -0.03310046, 0.008783119, -0.0055746734, 0.010587869, -0.006991737, -0.014237476, 0.013147941, 0.019277409, 0.029490959, 0.014946007, -0.00782727, -0.0068781045, 0.015587697, -0.028662112, -0.030774336, -0.014478109, -0.009043805, -0.004334743, -0.033875834, -0.007513109, 0.011396665, -0.00271381, -0.043982435, 0.013000888, 0.019638358, 0.019865623, -0.02621567, 0.0482871, 0.014611795, -0.032592457, -0.03834092, -0.021763954, 0.03491858, -0.0012858846, -0.013629208, 0.004588745, -0.024424288, -0.0180074, -0.03285983, 0.021095527, -0.035587005, -0.018408455, -0.00050424394, -0.010895345, 0.008642749, 0.0072056334, -0.032378558, 0.017472658, -0.009986286, 0.0019484621, 0.012232197, 0.016576966, -0.052324396, -0.03491858, 0.039838195, -0.033982784, -0.036282167, -0.006045914, -0.004846089, 0.009531756, 0.0074596354, -0.0008981976, -0.0068313144, 0.011724194, -0.016804231, 0.035747427, 0.020854894, -0.0070318426, -0.022579433, -0.00583536, 0.02097521, -0.029089903, -0.03906282, 0.05743117, 0.0032669324, 0.011303085, -0.022833435, -0.054142512, 0.023221122, 0.013769577, 0.011443455, 0.013368522, -0.03887566, -0.0018749352, -0.020774683, -0.001970186, 0.04232474, -0.0066441554, -0.013442049, -0.04938332, -0.04285948, 0.011403349, -0.007660163, -0.0007235712, -0.0029327194, 0.014611795, -0.017151814, 0.0010084879, 0.025186295, 0.018288137, -0.03176361, -0.0068714204, 0.030988233, -0.0022860172, -0.0021991218, -0.011430087, 0.008268431, 0.0035927903, 0.006841341, -0.048902053, 0.013889894, -0.027178206, -0.047431514, 0.0066508395, -0.0060091508, 0.016991392, 0.013368522, -0.029196851, 0.00451856, 0.035079002, -0.034062993, -0.015814962, 0.013970105, 0.0005861261, -0.010641343, -0.042191055, -0.0012207131, -0.0024464396, 0.029250326, 0.04267232, -0.003333775, 0.028207581, -0.006924894, 0.035587005, -0.016924549, -0.0008430524, -0.008214957, -0.027806526, -0.02360881, 0.038554817, -0.02514619, 0.009611967, -0.0036696591, -0.0035226054, -0.025440296, -0.017151814, 0.024892187, -0.010013022, 0.008148114, 0.012633253, -0.029597906, -0.0012842136, -0.0060091508, -0.01763308, -0.0031399315, 0.009471598, 0.012138618, -0.022779962, -0.017913818, 0.040854204, -0.023662284, -0.04096115, 0.00836201, -0.034731418, -0.009825864, 0.00057609973, -0.036041535, 0.022178378, -0.002252596, 0.00021191196, -0.029597906, 0.027485682, -0.0052304342, 0.023033964, -0.0032919985, 0.03165666, 0.011289717, 0.028608637, 0.006273179, 0.014946007, 0.015814962, -0.059890978, -0.014892533, -0.0029160087, 0.011597193, 0.02523977, -0.019772043, -0.02296712, -0.011537034, -0.01049429, 0.020814788, -0.0034357102, 0.0058520706, -0.005531226, -0.050853856, -0.006146178, 0.017298868, 0.023956392, 0.024397552, 0.03419668, 0.008936857, 0.016242754, -0.019852255, 0.014678637, -0.013415311, -0.02088163, -0.024651553, 0.0044985074, 0.05705685, 0.03997188, 0.044891495, 0.007346003, -0.014678637, -0.016617073, 0.012379251, -0.006226389, 0.0042645587, 0.0011839498, 0.010267025, 0.000997626, 0.004692351, 0.014464741, 0.013528944, 0.011095873, 0.0061294674, 0.037565548, 0.018034136, -0.004511876, 0.011283033, 0.017579606, -0.02216501, 7.483239e-05, 0.015975384, 0.017846977, -0.0051168017, 0.014478109, 0.032726143, -0.024972398, 0.025012504, 0.020574154, -0.042806007, 0.024758503, -0.012573095, -0.008301852, 0.017499395, -0.0013761222, -0.0079475865, -0.030881286, -0.003250222, -0.002969483, -0.012426041, 0.0047224304, -0.0005456028, -0.0007691078, 0.043207064, 0.018769404, 0.010126655, 0.0036529487, 0.07021148, -0.014317687, -0.014932639, 0.010079865, -0.0010051457, -0.010868608, -0.004866142, 0.016777495, 0.059837505, 0.0023578731, 0.018435191, 0.012486199, -0.010547764, 0.013823052, 0.04312685, -0.006273179, -0.023943022, 0.0030797732, -0.012479515, -0.031068444, 0.014865796, 0.046522457, -0.00013065641, -0.0074930564, 0.008642749, 0.0029143377, -0.021657005, -0.0024280578, -0.012412673, -0.014852428, -0.0018582245, 0.009398071, 0.0027087966, -0.01066808, 0.0019818833, -0.006263152, -0.0194512, -0.015694644, -0.020828158, 0.0076200576, 0.02315428, 0.03355499, 0.0021038712, -0.019504674, -0.04072052, -0.0066174185, 0.0024213735, -0.028715584, 0.024651553, -0.019330883, -0.0020153048, -0.015708013, 0.015226746, -0.010848556, 0.027833262, 0.009665442, 0.040586833, -0.026483042, 0.036415853, 0.0027539155, 0.00023018924, -0.017312236, -0.017232025, 0.01438453, -0.0018899748], "content_hash": "6a3cdaa616edc03a2d22ed59b3d0476b"}, {"content": "together to complete patterns like \"A B ... A\" \u2192 \"B\" (if the model has seen \"A B\" before, when it sees \"A\" again, it predicts \"B\"). In the dashboard, you can start to identify circuits by ablating combinations of heads and seeing which combinations have outsized effects. ### Superposition\n\n**Superposition** is the idea that neural networks represent more features than they have dimensions. A 768-dimensional embedding might encode thousands of different concepts by overlapping them. This makes interpretation challenging because a single neuron can participate in many features.\n\n### Induction Heads\n\n**Induction heads** are one of the best-understood circuits. They are pairs of attention heads (typically one in an early layer and one in a later layer) that work together to copy patterns from context. If the model has seen \"Harry Potter\" earlier in the text and encounters \"Harry\" again, induction heads help it predict \"Potter.\"\n\nYou might observe induction-like behavior in the dashboard when using prompts with repeated patterns.\n\n### Polysemanticity\n\nNeurons and heads are often **polysemantic** -- they respond to multiple unrelated features. An attention head might handle both pronoun resolution and list formatting, depending on the input. This is why head categories are approximate: the same head may behave differently for different prompts.\n\n## Notable Research Groups\n\nThese organizations have published influential work in mechanistic interpretability:\n\n- **Anthropic**: Published foundational work on transformer circuits, superposition, and dictionary learning for interpreting neural networks\n- **EleutherAI**: Open-source AI research group that has contributed tools and analysis for model interpretability\n- **Redwood Research**: Focuses on alignment-relevant interpretability, including causal interventions on model behavior\n- **DeepMind (Google)**: Research on understanding internal representations and how models store knowledge\n\n## Further Reading", "source_file": "mechanistic_interpretability_intro.md", "chunk_index": 1, "embedding": [0.011560152, -0.010602332, 0.00859091, 0.011626462, 0.008929831, 0.0013787084, 0.021499377, 0.015722986, -0.030267116, 0.01396207, 0.020187901, -0.05605458, -0.015619836, -0.012856893, -0.0015739563, -0.015104086, -0.008907727, 0.032801654, 0.037428662, 0.0497477, -0.012260097, -0.033597384, 0.042409327, 0.068078905, 0.025021208, -0.022295104, 0.003945482, 0.09336536, 0.026259007, 0.024446517, 0.030267116, -0.027894668, -0.046211135, -0.020585764, -0.04948246, 0.018301733, -0.010432871, 0.0125032365, -0.028174646, 0.07161547, 0.03648558, -0.047242634, -0.02994293, 0.020099487, -0.014153634, 0.01920061, -0.023798145, -0.029456653, -0.03032606, 0.08428817, -0.02854304, 0.03418681, -0.018846953, 0.0030613404, -0.010904414, 0.019627944, -0.04845096, 0.011309645, -0.008369874, -0.017889133, 0.003177384, -0.019318495, 0.02874934, 0.02720209, -0.041053645, -0.012311673, -0.04845096, 0.019554267, -0.006258986, 0.021882506, 0.046800565, 0.027393656, -0.0021035203, 0.015295651, 0.0057063974, 0.0003941798, -0.022412991, 0.03890223, -0.02505068, -0.030090287, 0.0044575473, -0.009246648, 0.029869251, -0.066546395, -0.03097443, -0.01612085, -0.022250898, -0.02474123, -0.06725371, -0.05962062, 0.013343171, -0.018832216, -0.025198037, 0.012289569, 0.0060011116, 0.013822081, -0.007964643, -0.013512632, 0.017962812, 0.03459941, -0.01252534, -0.026509514, -0.013343171, -0.008730899, 0.023282396, -0.0039381143, -0.009732926, -0.042704042, -0.014404141, -0.03518884, -0.06725371, 0.009526626, -0.017564947, 0.002352185, 0.021838298, -0.028292533, -0.06253829, -0.010226572, 0.0053343214, -0.0023098201, -0.013954702, -0.02251614, 0.0208068, 0.018699596, 0.00042203948, -0.045886952, 0.007824654, -0.024078123, 0.016135585, -0.010727585, 0.010617067, 0.015575629, 0.013888392, -0.04821519, -0.009401373, -0.03442258, -0.030414473, -0.01464728, -0.014897787, -0.029618746, 0.028528303, -0.012952675, -0.0020519453, -0.00036977383, -0.027261034, 0.024652816, 0.024770701, 0.027423127, -0.010602332, 0.015251444, -0.013394746, -0.036397163, -0.031770155, -0.028425153, -0.035837207, -0.028012555, -0.00797201, 0.052311715, 0.028395683, -0.052518014, 0.029589273, 0.03763496, -0.016887106, 0.0134757925, -0.024063388, 0.054757837, -0.071674414, -0.0011401743, -0.028012555, -0.047920477, -0.014153634, -0.008900359, -0.014536763, 0.0027850461, 0.046800565, -0.032624826, -0.055907223, -0.036839236, -0.04081787, -0.02388656, 0.007986746, 0.0098655475, -0.051073916, 0.024785437, -0.029589273, -0.017402854, -0.032565884, 0.038253862, -0.019583737, 0.0071504954, 0.0037189208, 0.038342275, 0.057675507, 0.02005528, 0.028115705, -0.050278187, 0.08841416, -0.0028789863, 0.026435835, -0.055907223, 0.0025879564, 0.02422548, 0.060062688, 0.01984898, 0.03076813, 0.025478015, 0.011928544, 0.020276316, -0.044737566, -0.03230064, 0.027835727, -0.026848435, 0.046004836, 0.004369133, -0.06578014, -0.036986593, -0.051457044, 0.014234681, -0.003470256, 0.045916423, 0.02357711, 0.011655934, -0.0047006863, 0.00886352, -0.016739748, 0.053814754, 0.027732575, -0.021794092, 0.014514659, -0.00192485, 0.05947326, 0.007669929, 0.005864806, -0.0010968882, -0.029456653, -0.05861859, 0.043794483, 0.0006046239, 0.028587246, -0.023636052, 0.029766101, 0.016916577, -0.0017074986, 0.050572902, 0.033420555, 0.04839202, -0.008635117, -0.019510059, -0.0038939072, 0.02590535, 0.05543568, 0.041672543, -0.028115705, 0.019053252, -0.016061906, -0.020335257, -0.024638081, -0.016901841, -0.024608608, 0.0058095474, -0.020777328, 0.0264211, -0.012724272, -0.0110001955, -0.026435835, -0.024505459, -0.0310039, 0.003908643, -0.037988618, -0.04414814, -0.008642484, -0.00052680104, 0.020983629, 0.012584283, -0.07668455, -0.051545456, 0.009246648, 0.016887106, -0.029279824, 0.0015582996, -0.01659239, 0.014212577, 0.04777312, 0.014418877, -0.036190864, 0.009556098, -0.020615237, -0.022250898, -0.043440826, -0.03097443, 0.005341689, -0.020688914, -0.005735869, -0.014087323, -0.0034942015, -0.018316468, 0.02303189, 0.038165446, 0.04497334, -0.0053159017, 0.02292874, -0.057852335, -0.0012276676, 0.018876424, 0.022133013, 0.019023782, -0.02197092, 0.003374474, 0.033243727, -0.020659443, -0.009688719, -0.012908468, 0.026642134, 0.0057579726, -0.053785283, -0.015988229, 0.061182603, 0.006793155, 0.008296195, -0.041554656, -0.026332686, -0.013976806, 0.024107596, -0.004048632, 0.007927803, 0.01731444, -0.03545408, -0.0053195856, -0.032536414, -0.015973492, 0.047684707, -0.026155857, 0.004763313, 0.011611727, -0.04762576, -0.06436551, -0.03757602, 0.024137067, -0.034894124, 0.015384065, -0.08098737, -0.008598277, -0.040847342, 0.010727585, -0.012208522, 0.084347114, -0.027157884, 0.030163966, 0.025816936, 0.029751366, -0.023297133, -0.044089198, -0.008671956, -0.046653207, 0.014868315, 0.03734025, -0.037016064, -0.005153809, 0.009482419, 0.0056990297, -0.036603466, 0.0070436615, 0.051928587, 0.031269144, -0.041289415, 0.0042991387, -0.04226197, 0.021705678, -0.0062295147, 0.018095432, 0.012532708, -0.002748207, -0.013556839, 0.0017636784, -0.052193828, 0.025964294, -0.03315531, 0.00044943867, -0.046918448, 0.053490568, 0.021617264, 0.00010574012, -0.02313504, -0.036986593, 0.009968697, 0.035336193, 0.016503977, -0.019053252, -0.045680653, 0.017152349, 0.00298582, -0.071674414, 0.004111259, 0.010705481, -0.016901841, 0.027747313, 0.0052164355, -0.0066789533, 0.057646036, 0.05072026, 0.015708249, -0.01974583, -0.029751366, 0.008716163, 0.041230474, 0.037487607, 0.03501201, 0.007356795, 0.0010443924, 0.0022932424, 0.046623737, -0.024181275, 0.0038754875, 0.0008726294, -0.03424575, -0.025345394, 0.022943476, -0.005010136, 0.045297522, 0.055347268, 0.022162484, -0.056850307, -0.027806254, -0.026126385, -0.033626854, 0.025198037, 0.050572902, -0.06707688, 0.029441917, -0.05970903, -0.0042659836, 0.059797447, 0.04523858, -0.008450921, 0.043558713, -0.035129894, 0.06725371, 0.01697552, -0.022737175, -0.018198581, 0.040906288, -0.03527725, -0.027054735, -0.020983629, -0.016798692, 0.01995213, -0.013497896, -0.070613444, -0.009106659, -0.06507283, 0.022015126, 0.009327695, 0.007047346, 0.019701622, -0.013903127, -0.012201155, 0.069139875, -0.0023263977, 0.059797447, -0.047183692, 0.023002418, 0.017668096, -0.024166537, -0.0047375257, 0.022737175, 0.005864806, -0.023400282, -0.02888196, 0.023709731, 0.0036286647, -0.005113286, -0.027806254, 0.026465306, -0.021248871, -0.002265613, -0.022457197, 0.032389056, 0.016091378, 0.033803683, -0.034098394, -0.013888392, -0.041348357, -0.002840305, 0.008296195, 0.025610637, 0.062361456, -0.041908313, -0.012974778, 0.03179963, 0.036456108, -0.026612664, -0.03946219, 0.03291954, -0.018404882, 0.026597928, -0.01824279, -0.04547435, -0.04246827, -0.0073973183, -0.0068299943, 0.057999693, -0.050189774, -0.029883988, 0.029648216, -0.006023215, 0.018316468, 0.017270233, 0.016916577, -0.049335103, 0.027334712, -0.01454413, 0.0012322724, 0.00595322, 0.014079955, -0.018994309, 0.014949362, 0.021219399, 0.008915095, 0.018714331, -0.0047006863, -0.017535476, -0.019996338, -0.04329347, 0.0015131716, 0.03424575, -0.022118276, -0.024019182, -0.04759629, -0.054816782, -0.006590539, 0.006885253, -0.016386092, 0.007139444, 0.0095192585, -0.042144086, -0.004332294, 0.0016955258, -0.030620772, -0.04102417, 0.0106760105, 0.004487019, 0.030443944, 0.008030953, 0.020983629, -0.016179793, 0.015929285, 0.0073604793, -0.03651505, 0.0068299943, 0.060357403, -0.006697373, -0.017122876, 0.026789492, -0.0528422, -0.026244272, -2.7701378e-06, -0.013984174, -0.008907727, 0.03097443, -0.011515945, -0.008870888, -0.046918448, 0.0020206321, -0.006266354, -0.0026597928, 0.026804227, 0.019686887, -0.011287542, 0.031269144, -0.037428662, -0.016327148, -0.025522223, 0.044649154, -0.012223259, 0.0076036183, -0.019583737, -0.008598277, 0.006642114, -0.014249416, -0.0008376321, -0.03949166, -0.035336193, -0.02429916, 0.010292882, -0.033508968, 0.014941994, -0.009725558, -0.009246648, -0.02368026, -0.0020832587, 0.01242219, -0.018419618, -0.0029545065, 0.015384065, -0.030473415, -0.012739007, 0.0013676566, -0.027762048, -0.04394184, 0.044089198, -0.01321055, 0.009489787, -4.371781e-05, -0.008266725, -0.01013079, 0.000867564, -0.024593873, 0.019893186, 0.033273198, 0.027437862, -0.014802004, -0.0050359233, 0.0040117926, 0.00010556744, -0.058294404, 0.017579682, -0.011935912, 0.010189733, 0.015457743, -0.014772533, 0.026568456, -0.017800719, 0.001538038, -0.018286996, 0.029382974, 0.009106659, 0.022029862, 0.003989689, 0.002939771, -0.023415018, -0.032389056, -0.002744523, 0.04205567, 0.007272065, -0.01659239, 0.0005641008, -0.004708054, -0.021676205, 0.055759866, -0.015899815, -0.0023908664, -0.02645057, -0.016106114, 0.0035660379, 0.01710814, -0.017992282, -0.018301733, 0.008642484, -0.016916577, -0.02409286, -0.03200593, -0.030031344, 0.024903324, 0.019627944, -0.0034813078, -0.016577655, -0.0011797765, -0.012709536, -0.01184013, 0.015855607, -0.03004608, -0.015118822, -0.008870888, -0.027452597, -0.013070561, 0.00859091, 0.03518884, 0.025139093, 0.009401373, 0.009482419, -0.04520911, 0.017638626, 0.044501796, 0.042379856, -0.0015233024, -0.017388118, 0.011206496, -0.010138158, -0.004100207, 0.010690746, 0.022560347, 0.026907377, -0.030591302, -0.016223999, 0.0043765013, 0.00865722, 0.014838844, 0.0030631823, -0.0062847733, -0.010565492, -0.0052716946, -0.013203182, -0.014735694, -0.026715813, -0.007261013, -0.031917512, 0.006785787, 0.006789471, -0.031770155, -0.05970903, -0.009570833, -0.00020215531, -0.018360674, -0.011781188, -0.0013345013, -0.026671607, 0.0033468446, 0.02857251, 0.020821536, -0.030090287, -0.08216623, 0.018979574, 0.01352, 0.035336193, 0.002026158, 0.013718931, -0.00046210215, -0.0027684686, 0.0033965774, -0.038931705, -0.052989557, 0.012392718, -0.048244663, -0.012834789, 0.022737175, -0.016798692, -0.025183301, -0.013365274, -0.040464215, 0.03784126, -0.015472479, 0.009025613, -0.004472283, -0.029486123, 0.02432863, 0.02269297, -0.015295651, 0.023400282, -0.034510996, -0.028970374, -0.009349798, 0.0027611007, 0.0021440436, 0.02429916, 0.022309842, 0.021720413, -0.012444294, 0.040022146, -0.027334712, -0.026214799, 0.00041467164, 0.014809373, -0.0012460871, 0.003713395, 0.0056585064, 0.006303193, -0.0012313514, -0.013387378, 0.035807736, 0.0164303, 0.0030189753, 0.034540467, -0.020865742, 0.013394746, -0.010108686, -0.012871629, 0.041348357, 0.0026542668, -0.015575629, 0.025507485, 0.015384065, -0.0019045884, 0.016577655, -0.0073604793, 0.011707509, -0.024623346, 0.036780294, 0.018449089, -0.001972741, 0.00886352, -0.010388664, -0.0019616892, 0.018110167, -0.011206496, 0.0026763703, 0.0020648392, -0.0158114, -0.050455015, -0.0065794876, -0.08092843, -0.038077034, 0.01615032, 0.008318299, -0.022015126, 0.002554801, -0.018832216, -0.01666607, -0.11924124, 0.0016080326, 0.008141471, -0.009777133, -0.0022195638, -0.016798692, 0.051339157, 0.029751366, 0.021278342, -0.03928536, 0.021926712, -0.018110167, 0.014411509, 0.015045144, 0.03527725, -0.050042417, -0.024181275, -0.027983082, 0.018198581, -0.0034813078, -0.014234681, 0.014698855, 0.0012387193, -0.0011816184, -0.045621708, -0.0088266805, -0.02292874, 0.0073973183, 0.003468414, 0.006550016, -0.044884924, -0.022354048, -0.051869642, 0.018110167, -0.0022490353, 0.016061906, 0.030709187, -0.0071173403, 0.00859091, -0.010138158, -0.0057579726, -0.021307813, -0.001108861, -0.017977547, 0.027850462, 0.010749688, 0.016990256, -0.0076772966, -0.0076404572, 0.036131922, 0.0001353842, -0.020025808, -0.0037723377, -0.0026119018, 0.0049548773, 0.006424763, -0.0060858415, -0.008745634, 0.036898177, 0.0106760105, -0.009276119, 0.026229536, 0.048981447, 0.025993764, -0.00055627245, 0.023002418, 0.0121569475, 0.011700141, 0.002359553, -0.016165057, -0.0056585064, 0.01752074, -0.037104476, 0.023046626, -0.008981406, -0.018802745, 0.026391627, -0.004479651, 0.0074341577, -0.015781928, 0.06153626, -0.0035899836, -0.047714178, 0.02704, 0.00940874, 0.027953612, -0.011958016, -0.013114768, -0.015251444, 0.04730158, 0.0077730785, -0.010771792, -0.0061300485, -0.013232653, -0.008627749, 0.019922659, 0.008841417, 0.026067443, 0.0636582, -0.018714331, 0.043617655, 0.038931705, -0.040906288, -0.040169504, -0.042586155, 0.027879933, -0.02771784, -0.0008058583, 0.0070068226, 0.02186777, 0.005691662, 0.0026837382, -0.012775847, -0.030237645, 0.017270233, 0.024151802, 0.025109623, 0.012039062, 0.003960218, -0.0019045884, 0.014043116, -0.03209434, -0.015236707, -0.007003139, -0.026096914, 0.0034942015, 0.0020979943, 0.014315727, -0.0076330896, -0.009902386, -0.03336161, -1.1562627e-05, -0.004656479, 0.021293078, 0.0101676285, -0.002361395, -0.011037035, 0.034334168, -0.042586155, 0.0027058418, -0.023429753, -0.022265634, 0.018994309, -0.0015711933, -0.005157493, 0.00093571655, -0.024623346, -0.019097459, -0.019377438, 0.0031460708, 0.026583193, 0.016268207, -0.0022858744, 0.028852489, -0.0017268391, 0.029176675, 0.026155857, -0.047861535, -0.039373774, 0.053785283, 0.0006359373, 0.028248325, 0.019451115, -0.013866288, 0.0021403595, -0.010116054, -0.03887276, -0.004078103, -0.007419422, 0.021499377, -0.02665687, -0.020644708, 0.020821536, 0.03247747, 0.011073874, 0.024623346, 0.0009274277, 0.013195814, 0.010683378, 0.01163383, 0.0030981796, 0.03501201, -0.008664588, 0.012952675, 0.011707509, 0.013512632, -0.001970899, -0.0040265284, -0.026376892, 0.0032971115, 0.0025861142, 0.012429558, -0.024652816, 0.02186777, -0.0024295475, -0.0036010353, 0.0059163813, 0.025035944, 0.0046527954, -0.0019211661, -0.020349994, 0.035984565, -0.006712109, -0.034923594, -0.026627399, 0.048893034, 0.016901841, -0.0151925, -0.0129895145, -0.008244621, -0.016872369, 0.01516303, 0.0160177, 0.015030408, 0.0052127517, 0.019790037, 0.016165057, 0.024343366, 0.005540621, -0.009283488, -0.017388118, -0.026023235, -0.029471388, 0.0022324577, -0.0013446321, 0.014603073, -0.025507485, 0.02375394, 0.010219204, -0.015384065, 0.003904959, 0.014300991, 0.00759625, 0.029338766, 5.7935453e-05, -0.0013906811, 0.010757057, 0.0006893542, 0.041908313, 0.0020427357, 0.025684314, 0.018773274, -0.025094887, -0.03542461, -0.007264697, -0.02334134, 0.0060858415, 0.022250898, 0.0035347247, -0.015870342, -0.03333214, 0.021440435, 0.04738999, -0.015840871, -0.0015804032, -0.014838844, 0.03993373, 0.042144086, 0.019627944, 0.015472479, 0.025522223, 0.025301186, 0.005481678, -0.032978483, -0.010174997, -0.013321067, -0.0030429207, -0.02398971, -0.0053969477, 0.00033523704, -0.045621708, 0.040434744, -0.0033394767, 0.010661274, -0.011169656, -0.0032952696, -0.029677687, -0.00058067846, -0.01046971, -0.010690746, 0.0067710513, 0.00828146, 0.00886352, 0.0147946365, 0.024417045, -0.01375577, -0.037045535, -0.008060425, 0.0044022887, -0.02877881, -0.022280369, -0.017034462, 0.01433783, -0.025021208, -0.024829645, 0.0133136995, -0.023282396, 0.009062452, 0.016179793, -0.028425153, 0.022958212, 0.011862233, 0.008649852, -0.032801654, 0.023798145, -0.009460315, 0.013939966, -0.062832996, -0.015104086, 0.0049880324, -0.025374865, 0.027319977, 0.02655372, 0.0044059725, 0.0208068, 0.022663498, 0.011729612, -0.053254798, -0.017388118, 0.00045427383, -0.0028274113, 0.020968892, 0.012053798, 0.027408391, -0.0032436945, -0.031269144, 0.01594402, -0.015546157, 0.021749884, 0.0038902233, -0.020968892, -0.008377242, -0.0087087955, 0.010978092, -0.031917512, 0.030414473, -0.011037035, -0.021911977, 0.013063192, 0.013556839, 0.009482419, 0.0019580054, 0.009762397, 0.0024737546, 0.004674899, -0.004951193, -0.005864806, 0.0058537545, 0.0074894163, 0.024623346, 0.056791365, -0.02088048, -0.0076478254, -0.005684294, 0.013387378, 0.016223999, -0.034481525, 0.011655934, -0.028439889, -0.019686887, -0.0405821, 0.009217177, -0.016179793, -0.003273166, 0.0060895258, 0.0019635311, -0.023857089, -0.018640652, 0.020335257, 0.016710278, 0.025315922, 0.028955638, -0.01156752, -0.022958212, 0.0046638474, 0.027143149, 0.013380011, 0.022575084, 0.016503977, 0.02916194, -0.011007563, -0.030178702, -0.0076036183, 0.035070952, -0.013107399, 0.013100032, 0.0011291226, -0.011420163, -0.03445205, 0.015708249, -0.010204468, 0.0034868335, -0.0015279073, -0.036426637, 0.0067710513, 0.02833674, -0.009401373, 0.023017153, -0.014529394, -0.008310932, -0.024903324, 0.020349994, -0.014426244, -0.0065094926, 0.011191759, -0.009939225, 0.004372817, 0.04986559, -0.014831476, 0.015899815, -0.024004446, 0.054875724, -0.015487215, 0.008841417, -0.02088048, -0.031416498, -0.037693907, 0.009099291, 0.015310386, 0.017476533, -0.014536763, -0.0028329373, 0.023901295, 0.002884512, 0.014138899, 0.009261384, -0.011545416, 0.012635858, 0.034068923, -0.009622408, -0.010071847, -0.012599018, -0.018935367, -0.002352185, -0.010757057, -0.017388118, 0.023606582, -0.011678037, -0.03097443, 0.016356621, 0.015958756, 0.025610637, -0.0023890245, 0.039550602, 0.013836817, -0.019775301, -0.004579117, -0.014757797, 0.020482615, -0.004615956, -0.015207237, 0.0051759128, -0.027599955, -0.008598277, -0.022265634, 0.008568806, -0.03966849, -0.011081242, -0.0022527191, -0.003945482, 0.0042733513, -0.012002223, -0.009747662, 0.007662561, -0.010705481, -0.015207237, 0.008878255, 0.02748207, -0.042615626, -0.04547435, 0.010734953, -0.044266026, -0.010661274, -0.0033118473, 0.019009044, 0.018051226, 0.011221231, -0.025006473, -0.022133013, 0.0007390872, 0.0025713786, 0.045120694, 0.00965188, -0.018360674, 0.0077730785, 0.01892063, 0.04479651, -0.020482615, -0.047036335, 0.053431626, 0.0077730785, 0.029618746, -0.02802729, -0.012945307, 0.024372838, 0.012915836, 0.019863715, 0.015560893, -0.021705678, -0.009681351, -0.024564402, -0.005787444, 0.025212772, 0.003031869, -0.0022711388, -0.049394045, -0.014072588, -0.0059053292, -0.010808632, -0.031062843, 0.0055037816, 0.018021753, 0.008649852, -0.0016817111, 0.022280369, 0.007552043, -0.019672152, -0.008134103, 0.0037649698, -0.00046348362, 0.014418877, -0.018522767, 0.0210131, -0.008251988, 0.00742679, -0.039403245, 0.017461797, -0.011707509, -0.013350539, 0.009239281, -0.018095432, 0.008303564, -0.0026211115, -0.02104257, -0.0021495693, 0.029014582, -0.013527367, -0.0036820816, 0.0036876074, -0.004862779, -0.018316468, -0.04420708, -0.0153988, 0.002742681, 0.015855607, 0.062479343, -0.01865539, 0.032654297, 0.009423477, 0.011228599, 0.0052606426, -0.019966865, -0.011118081, -0.034569938, -0.019244816, 0.01943638, 0.010454975, 0.01813964, -0.00022667642, 0.0032915857, -0.0032694822, -0.022914004, 0.002552959, 0.0028808282, -0.0020666812, 0.0069147246, -0.008694059, 0.0034831497, -0.015487215, -0.011434899, -0.012186419, -0.016621863, 0.020733122, 0.011582255, -0.039845318, 0.014072588, -0.033450026, -0.023356074, 0.0046380595, 0.022707704, 0.019672152, -0.023252925, -0.0125032365, 0.025772728, -0.00501382, -4.856735e-07, -0.040316857, 0.014197841, 0.005058027, 0.029515594, -0.009114027, 0.029780837, 0.0023945502, 0.03395104, -0.02942718, 0.026244272, 0.02686317, -0.025198037, -0.031946983, 0.01612085, -0.00070639234, 0.014603073, -0.031328086, -0.020423671, -0.0104034, -0.037929676, 0.015516686, 0.009261384, 0.026538985, -0.0040596835, -0.056820836, -0.013136871, 0.002687422, 0.0041812533, 0.0026745284, 0.033420555, -0.0030521306, 0.011405427, -0.0055664084, -0.0049990844, -0.00759625, 0.0018603813, -0.04205567, -0.009290855, 0.020349994, 0.021956185, 0.02932403, 0.017653361, 0.0036231389, -0.024932794, 0.0024977, -0.030827072, -0.008134103, -0.0048038363, 0.027275769, 0.025183301, 0.019274287, 0.008568806, 0.02104257, 0.014433612, 0.006686321, 0.055907223, 0.016209263, -0.028484097, 0.02751154, 0.0012175367, -0.02093942, -0.021941448, -0.003941798, 0.030502887, -0.010285514, -0.0031460708, 0.030886015, 0.0023337656, 0.030532358, 0.023149775, -0.023945503, 0.036220334, -0.00578376, -0.015634572, 0.020762593, -0.011655934, -0.027909404, -0.018773274, 0.010609699, -0.0017056565, 0.004461231, 0.01279795, -0.010749688, 0.018316468, 0.021808827, 0.031180728, 0.027231563, 0.012591651, 0.04544488, -0.023503432, -0.018228054, -0.001153989, 0.0055627245, -0.010609699, 0.011479106, 0.011921177, 0.051280215, -0.0029342452, 0.029014582, -0.017668096, -0.019009044, -0.006354768, 0.05499361, -0.00041029698, -0.018493297, 0.0015932969, -0.002024316, -0.027261034, 0.0070620812, 0.033273198, 0.014212577, -0.0050027682, 0.0067636836, 0.02419601, -0.02388656, -0.005010136, -0.02546328, -0.0067526316, 0.015634572, 0.016209263, -0.01868486, -0.011626462, -0.02248667, -0.0038423322, -0.031475443, -0.0045496454, -0.019185873, 0.0032510625, 0.011228599, 0.0060084793, 0.015177765, -0.021646734, -0.02706947, -0.024358103, -0.007128392, -0.0022398254, 0.041702013, 0.00944558, -0.030163966, -0.037988618, -0.022162484, -0.01827226, 0.0026616347, -0.017122876, 0.016356621, -0.021499377, 0.01892063, -0.016725013, 0.0049548773, 0.00491067, -0.008811945, -0.00043723566, 0.004627008], "content_hash": "4cd25d11b4ff6033d61b01ed146c1ea8"}, {"content": "dictionary learning for interpreting neural networks - **EleutherAI**: Open-source AI research group that has contributed tools and analysis for model interpretability - **Redwood Research**: Focuses on alignment-relevant interpretability, including causal interventions on model behavior - **DeepMind (Google)**: Research on understanding internal representations and how models store knowledge ## Further Reading\n\nIf you want to explore the research behind this dashboard's techniques:\n\n- \"A Mathematical Framework for Transformer Circuits\" (Elhage et al., Anthropic) -- foundational paper on how attention heads compose into circuits\n- \"In-context Learning and Induction Heads\" (Olsson et al., Anthropic) -- how models learn to copy patterns from context\n- \"Locating and Editing Factual Associations in GPT\" (Meng et al.) -- how facts are stored in MLP layers\n- \"Attention Is All You Need\" (Vaswani et al., 2017) -- the original Transformer paper\n\nThese papers are referenced here for context. The dashboard provides a hands-on way to explore many of the concepts they describe.", "source_file": "mechanistic_interpretability_intro.md", "chunk_index": 2, "embedding": [-0.028181022, -0.020663494, 0.0035840878, -0.013948183, 0.009160773, -0.0055863895, -0.005596094, 0.053153187, -0.03985195, 0.024208765, 0.010706977, -0.0510312, -0.017959256, -0.019835403, 0.03017362, -0.0033544214, 0.015293833, 0.0144139845, 0.003939909, 0.06407365, 0.027637586, -0.02195739, 0.03470225, 0.08032497, 0.021025786, -0.022177352, 0.003444994, 0.054602347, 0.0090119755, 0.012434326, 0.018425057, -0.013767038, -0.06158938, -0.011425088, -0.04093882, 0.007452832, 0.00059680885, 0.010804019, -0.030277131, 0.053929523, 0.02207384, -0.012000872, -0.03079469, 0.070646636, -0.035737365, -0.0087402575, -0.025049796, -0.0028320116, -0.04515692, 0.061848156, -0.029086748, 0.042853784, -0.04590738, 0.012790147, -0.03128637, 0.016950017, -0.03433996, 0.021271626, -0.018916737, -0.009697739, -0.015656123, -0.017441697, -0.00057699607, 0.028258655, -0.02869858, -0.026033157, -0.05408479, 0.0065406365, -0.0015437779, 0.0049362076, 0.05517166, 0.020818762, 0.012563716, -0.010739325, 0.019356662, -0.017001774, -0.043009054, 0.046373177, -0.043630123, -0.0141422665, -0.0061718766, -0.018683836, 0.01704059, -0.042853784, -0.01287425, -0.03348599, -0.00895375, 0.0005822525, -0.029552551, -0.038092252, 0.0033414825, -0.03912737, -0.025450904, -0.018567385, 0.013534136, 0.026524836, -0.0018130697, 0.005676962, 0.001976424, 0.03690187, -0.011974993, -0.042103328, 0.0068641105, -0.00809331, 0.01361177, -0.014543374, 0.03126049, -0.048728067, -0.015876085, -0.0008588225, -0.099164076, -0.013999938, -0.010687568, -0.009180182, 0.029578429, -0.026227241, -0.051678147, -0.0049523814, -0.011069267, -0.01938254, -0.030070107, -0.005592859, 0.051962804, 0.037005384, 0.004855339, -0.019162577, 0.028491557, -0.019718952, 0.026071973, -0.0090702, 0.005528164, 0.02784461, 0.0066085663, -0.06034724, -0.017726354, -0.04686486, -0.010875183, 0.007543405, -0.03547859, -0.036979504, 0.02048235, 0.009264285, -0.0230572, -0.003194302, -0.034546982, 0.010066499, 0.012828964, -0.006734721, -0.010072969, 0.04453585, 0.0072652176, 0.0028853847, -0.013870548, -0.037548818, -0.05260975, -0.028155144, 0.014194023, 0.029707817, 0.03444347, -0.06365961, 0.015837269, 0.058432274, -0.0019408418, 0.014012877, -0.05077242, 0.016613605, -0.020547044, -0.00047469753, -0.027327051, -0.030639421, -0.026887128, -0.019020248, -0.014530435, 0.01374116, 0.02220323, -0.02636957, -0.024972163, -0.014582191, -0.0076663247, 0.002781873, -0.026938884, 0.023238344, -0.033046067, 0.011877951, -0.028413923, 0.00084426615, -0.027456442, 0.012621941, -0.007543405, -0.0038913877, -0.02683537, 0.04818463, 0.06676496, 0.027508197, 0.012854842, 0.0033576563, 0.03620317, 0.02525682, 0.030277131, -0.047537684, 0.025722623, 0.0385063, 0.0257485, -0.020883458, 0.03175217, 0.026550714, 0.043707754, 0.0038816836, -0.03211446, -0.027301174, 0.033511866, -0.015850207, 0.020417655, 0.02243613, -0.057707693, -0.033149578, -0.0409647, 0.0069740913, 0.021970328, 0.048883334, 0.009853006, 0.03408118, 0.0067605986, 0.0019667195, -0.027378807, 0.04158577, 0.013779976, -0.026084913, 0.008203291, 0.009050792, 0.046813104, 0.0056057977, -0.002539268, 0.011949115, 0.02440285, -0.07613275, 0.024066437, -0.042387985, 0.023613574, -0.017312309, 0.021129297, -0.005576685, -0.016445398, 0.03348599, -0.0006340083, 0.02635663, -0.019654257, -0.03664309, 0.018399179, 0.05271326, 0.033796523, 0.021776244, -0.028853847, -0.019641317, -0.02709415, -0.0032832574, -0.014828031, -0.033304844, -0.020676434, -0.0015720818, 0.0075628133, 0.036694847, -0.0006166216, -0.02463575, -0.003181363, -0.029811328, -0.028517434, 0.016471276, -0.044820506, -0.04523455, 4.5463206e-05, -0.022358498, -0.00051796215, 0.013521197, -0.062158693, -0.034909274, 0.0038072846, 0.028595068, -0.0031053468, 0.011832665, -0.006466238, 0.0033932382, 0.04758944, 0.022397313, -0.023717087, 0.005460235, -0.013935244, -0.038092252, -0.043190196, -0.05480937, 0.009044322, -0.018463874, 0.032192096, -0.0463473, 0.001361824, -0.005262916, 0.017545208, 0.037186526, 0.048365775, -0.027016517, 0.0060036704, -0.04694249, -0.013883488, 0.007996268, 0.0189038, 0.014194023, -0.031053469, -0.00013454481, 0.07131946, 0.0025425027, -0.0064080125, -0.021556282, 0.028439801, 0.022138534, -0.03571149, -0.006453299, 0.04526043, -0.03190744, 0.019680135, -0.026382508, -0.044302948, -0.029500794, 0.017803987, 0.0006109608, -0.022397313, 0.02464869, -0.011386272, 0.00093079533, -0.042776152, -0.0020540575, 0.052557994, -0.0020944916, 0.010098847, 0.012796616, -0.05496464, -0.011192188, -0.03371889, 0.019654257, -0.05972617, -0.0014402663, -0.07597748, -0.013456503, -0.0738555, 0.004774471, -0.03180393, 0.062728, -0.029759573, 0.015759636, 0.0409647, 0.037859354, -0.030328887, -0.029759573, -0.006204224, -0.05041013, -0.007989799, 0.04515692, -0.063193806, -0.01899437, 0.029552551, 0.0021915338, -0.04981494, -0.024325216, 0.035219807, 0.0063724304, -0.019783646, -0.008067432, -0.007323443, -0.0013998321, 0.032036826, -0.0075628133, 0.02794812, 0.010377034, -0.024001744, -0.010008274, -0.050254863, 0.00619452, -0.011625642, -0.02587789, -0.04086119, 0.039179124, 0.013948183, 0.02257846, 0.0011911916, -0.033822402, 0.008248577, 0.021620978, 0.017868683, -0.051315855, -0.042232715, 0.05884632, 0.00037967716, -0.06241747, -0.028103389, 0.0046159686, 0.0033156045, 0.0014491618, 0.012304937, -0.020547044, 0.08001444, 0.05558571, -0.009710678, -0.0124019785, -0.04707188, -0.028413923, 0.042025693, 0.057448916, 0.011515661, 0.023652392, -0.004454232, -0.013831732, 0.0512641, -0.03180393, 0.0043216078, 0.004450997, -0.0060554263, -0.024596933, 0.027611708, 0.03296843, 0.018205095, 0.047744706, 0.0010432025, -0.009341918, -0.032554384, -0.011974993, -0.034132935, 0.041301113, 0.052661505, -0.051108833, 0.03633256, -0.06257274, 0.0031182857, 0.06298678, 0.033304844, -0.015539673, 0.040887065, -0.044924017, 0.05972617, 0.03876508, -0.004551274, 0.013999938, 0.031105224, -0.0333566, -0.04171516, -0.017286431, -0.015436161, 0.03961905, -0.02415701, -0.04500165, -0.022539642, -0.048883334, 0.048598677, 0.015384406, 0.01447868, 0.00069951173, -0.022229107, -0.014310473, 0.04148226, 0.013689403, 0.049323257, -0.008507356, 0.029707817, 0.011502722, -0.034650493, 0.028103389, 0.014090511, -0.009937109, -0.012816025, -0.012524898, 0.03690187, -0.018088644, -0.013779976, -0.041637525, 0.0155137945, -0.032295607, -0.0055022864, 0.009607167, 0.02771522, -0.022384375, 0.047770586, -0.04018836, 0.0059260367, -0.01607017, -0.010687568, -0.021349259, 0.03079469, 0.056051508, -0.051005322, -0.006414482, 0.036850113, 0.006466238, -0.034029424, -0.037341796, 0.023613574, -0.013100682, 0.02942316, -0.013779976, -0.048753943, -0.059312124, 0.0022254984, 0.009018444, 0.011347455, -0.016523033, -0.016277192, 0.07307916, -0.0017289665, 0.025502661, -0.0018567386, 0.002561911, -0.029630184, 0.020663494, -0.0018745296, 0.0053146714, -0.01631601, 0.022901932, -0.00656328, 0.012977762, 0.025528539, 0.0017694007, -3.214519e-05, 0.004328077, 0.0051011792, -0.020417655, -0.008733788, -0.0086496845, 0.036565457, 0.008429723, -0.030561788, -0.04171516, -0.05041013, -0.004502753, 0.016134864, -0.030251253, 0.012570185, 0.007090542, -0.056827847, -0.008766135, 0.0029727225, -0.03765233, -0.04231035, 0.015216199, 0.020870518, 0.033330724, 0.031545147, 0.02869858, -0.016251314, 0.0031279898, -0.00907667, -0.04168928, 0.01508681, 0.04526043, 0.029630184, -0.06329732, 0.019783646, -0.030717054, -0.020974029, -0.0074010766, 0.0108428355, -0.026783615, 0.025437966, -0.0019796586, -0.029656062, -0.021517465, -0.018360363, 0.0005988305, 0.006437125, 0.022707848, 0.01140568, -0.032192096, 0.011412149, -0.043630123, -0.001177444, 0.004525396, 0.014892725, 0.0030568258, 0.027508197, -0.01286778, -0.0041501666, 0.0045965603, 0.0043636593, 0.0076986724, -0.034365837, -0.04401829, -0.041896302, 0.00557992, -0.025839074, 0.04474287, 0.0071811145, -0.0022918107, -0.029267894, -0.0023516533, 0.023406552, -0.00027010049, -0.03775584, -0.018567385, -0.04293142, -0.017001774, 0.010454668, -0.023290101, -0.016950017, 0.034676373, 9.896271e-05, 0.0008361793, -0.019072006, -0.003545271, 0.01041585, 0.013560014, -0.0257485, 0.018386241, 0.0010925322, -0.011664459, -0.012324345, -0.02219029, -0.02414407, -0.0036326088, -0.02389823, 0.016393643, -0.014802153, 0.016587727, 0.01669124, 0.004441293, 0.0056510842, -0.023691207, 0.006818824, -0.03296843, -0.0031975368, 0.019356662, 0.0075239963, -0.0029775747, -0.028388046, 0.0038396318, -0.05703487, 0.01225965, 0.05136761, 0.0057545956, -0.014543374, 0.00723934, 0.0123955095, -0.04378539, 0.050694786, -0.017195858, 0.0008822743, -0.006631209, -0.030121863, 0.011593294, 0.03886859, -0.019059066, -0.014556313, 0.018955555, -0.039515536, -0.005492582, -0.043448977, -0.008423253, 0.022979567, 0.025942584, -0.002086405, 0.0030924077, -0.0022335853, -0.023949986, -0.012835434, 0.026628349, -0.03690187, -0.013805854, -0.0028028989, -0.020676434, -0.02221617, 0.0041501666, 0.020999907, 0.02683537, 0.025049796, 0.014297534, -0.056103263, 0.02085758, 0.01336593, 0.01249902, 0.016665362, -0.04779646, 0.0044477624, -0.019343723, -0.002285341, -0.006233337, 0.022035023, 0.021323381, -0.019136699, -0.02893148, 0.02353594, 0.036953628, 0.016600667, -0.0020637617, 0.01507387, -0.004007838, -0.0018405649, -0.007789245, -0.00857852, -0.007860409, -0.031338125, 0.003677895, 0.019589562, 0.03961905, -0.0090119755, -0.09212529, -0.009186651, 0.011056328, -0.034779884, -0.015410284, 0.015720818, -0.019848341, -0.01006003, 0.0049976674, 0.030975834, -0.038325153, -0.06805885, 0.009296631, 0.015811391, 0.029138504, 0.007174645, 0.02330304, 0.011470375, 0.012421387, -0.014297534, -0.016885323, -0.043474853, 0.018916737, -0.039774317, -0.007983329, 0.022138534, -0.02010712, -0.027559953, -0.0050106063, -0.034624618, 0.014310473, -0.016639482, 0.0007496501, 0.0035258625, -0.041119967, 0.025722623, -0.00821623, -0.013754099, -0.0008628659, -0.03798874, -0.020767007, -0.029449038, 0.0052338033, 0.0008082797, 0.0096848, 0.018334486, 0.027741097, -0.028129267, 0.020288266, -0.008352089, -0.053411964, 0.0036487824, 0.011075737, 0.00943896, -0.00014283381, -0.01385761, -0.00037785762, 0.026809493, -0.031622782, 0.045726232, -0.0015736992, -0.0033447172, 0.016768873, -0.016264254, 0.029707817, -0.004318373, -0.029474916, 0.028413923, 0.007789245, 0.005799882, 0.00980772, 0.002178595, -0.02526976, 0.033330724, -0.0068576406, 0.041637525, -0.033278964, 0.030251253, 0.02035296, -0.0024066437, 0.015565551, -0.025230942, 0.00142571, 0.018942615, -0.031726293, -0.0015825947, -0.010732855, -0.038894467, -0.031855684, -0.009678331, -0.03791111, -0.061278842, 0.0049006254, 0.0010496719, -0.0220609, -0.0013933626, -0.018929677, -0.017972194, -0.08487948, -0.0146857025, 0.025127431, -0.008035085, 0.029630184, -0.022371436, 0.020585861, 0.028362168, 0.016134864, -0.064642966, -0.0067735375, -0.008313272, 0.016561849, 0.011929708, 0.042051572, -0.049090356, -0.0020621442, -0.022720788, 0.026550714, 0.02035296, -0.012906598, 0.02686125, 0.022759603, 0.033046067, -0.059156857, -0.017972194, -0.015306772, 0.008604398, 0.025166247, 0.019162577, -0.033304844, -0.031726293, -0.04145638, 0.02477808, 0.016108986, 0.017946316, 0.021672733, 0.01238904, 0.0057545956, 0.005544338, -0.013023048, -0.027818732, 0.03775584, -0.015035054, 0.024131132, 0.024920408, 0.002319306, -0.014012877, -0.004929738, 0.018864982, 0.017713416, 0.0011345837, 0.00570284, -0.003040652, 0.004683898, -0.0008620572, -0.00035157538, -0.0064047775, 0.04083531, 0.009613636, 0.00051553606, 0.0049685547, 0.021517465, 0.020223571, -0.0057643, 0.019123761, 0.034805764, 0.022759603, -0.034754004, -0.03571149, -0.01139921, 0.009141364, -0.02304426, 0.029112626, -0.009050792, -0.014750397, 0.027559953, -0.0033705952, 0.016704177, -0.02414407, 0.044302948, -0.012227303, -0.046890736, 0.023121895, 0.020223571, 0.022539642, 0.0032832574, -0.027818732, -0.02722354, 0.03998134, 0.028388046, 0.018683836, -0.00820976, -0.012175547, 0.015927842, 0.027197663, 0.009393674, 0.044199437, 0.049607914, -0.014737458, 0.026188424, 0.03283904, -0.048365775, -0.02366533, -0.03214034, 0.029707817, -0.04523455, -0.011515661, 8.339554e-05, 0.0028546548, 0.005275855, 0.025476782, -0.005861342, -0.015850207, -0.0024357564, -0.0010051943, 0.01816628, 0.021996206, 0.02942316, -0.004745358, 0.027818732, -0.028750336, -0.03273553, -0.00820976, -0.0072134617, -0.018088644, 0.02488159, 0.0034158814, 0.0044445274, -0.007834531, -0.0035614446, -0.012880719, 0.014621007, -0.0041792793, 0.0017386707, 0.027534075, 0.01704059, 0.046036765, -0.022462009, 0.023730025, -0.03589263, -0.006304501, 0.0065406365, 0.0077439584, 0.0055346335, 0.00067646423, -0.012059096, -0.032580264, -0.004221331, 0.022772543, 0.01669124, 0.011004573, -0.014051694, 0.045545086, 0.026628349, 0.039774317, 0.020702312, -0.0730274, -0.023937048, 0.024493422, 0.013249479, 0.014698641, 0.0023662096, 0.0011758266, 0.008552643, -0.00521116, -0.017765172, -0.00269777, -0.029992474, 0.02463575, -0.028155144, -0.010034151, 0.0112051265, 0.028802091, 0.0037361204, 0.011425088, -0.0013246245, 0.041973937, 0.023277162, 0.026602471, -0.015733756, 0.0564138, -0.0019392243, 0.01630307, 0.033408355, -0.013378869, 0.017273491, -0.0021025785, -0.019084943, 0.0040046033, -0.006446829, -0.015177382, -0.011062798, 0.03348599, -0.01128276, -0.02207384, 0.01152213, 0.02391117, -0.01103692, -0.021012846, -0.049400892, 0.02906087, 0.009574819, -0.03633256, -0.010674629, 0.025929645, 0.01604429, -0.038894467, -0.010124724, -0.0064597684, -0.007866878, -0.009542472, -0.0014321795, 0.026809493, 0.018360363, 0.014401046, 0.031001711, 0.011800318, 0.008856708, -0.029086748, -0.027534075, -0.052687384, 0.0066635567, 0.0069611524, -0.021245748, 0.02488159, 0.002210942, 0.005748126, 0.038790956, -0.0029242015, 0.01630307, 0.010163541, 0.017558148, 0.027145905, 0.026408387, 0.002833629, 0.02415701, 0.0008466922, 0.03371889, 0.013637648, 0.022125596, 0.018179217, -0.008772604, -0.028595068, -0.016173681, -0.010687568, 0.017584026, 0.054912884, -0.006695904, -0.008934341, -0.0240535, 0.024312278, 0.03985195, -0.024972163, 0.0077310195, -0.023471246, 0.03998134, 0.028439801, 0.0041113496, -0.0040175426, 0.034262326, 0.033770647, 0.0053858357, -0.039929584, -0.016548911, -0.009426021, 8.5417254e-05, -0.011800318, -0.00082890113, 0.02415701, -0.033667136, 0.0441218, -0.002807751, -0.00070638553, -0.029630184, -0.022358498, -0.014038756, -0.01604429, 0.0018001307, -0.004806818, -0.012414917, -0.045545086, 0.0126736965, 0.0088308295, 0.016898261, -0.0043992414, -0.02895736, -0.027818732, 0.01679475, -0.012828964, -0.0407318, -0.011088676, -0.0059292717, -0.02722354, -0.03483164, 0.027068272, -0.015526734, -0.011748562, -0.0026395447, -0.011748562, 0.032295607, 0.01716998, -0.011218065, -0.037315916, 0.006818824, -0.036979504, -0.0019634848, -0.06329732, -0.015462039, 0.029086748, -0.02257846, 0.010603465, 0.02635663, 0.02256552, 0.038920347, 0.0017823396, -0.006029548, -0.044044167, 0.010512893, -0.0019311375, 0.008979628, 0.024260521, 0.0025716154, 0.017752232, -0.010409381, -0.022901932, 0.017558148, -0.02243613, 0.015811391, 0.009574819, -0.020340022, 0.014789214, -0.03421057, -0.004425119, -0.024700446, 0.0011232621, -0.0076986724, -0.020042425, 0.0058192904, 0.015358527, 0.0257485, 0.0010108551, 0.00723287, 0.00033216696, 0.005981027, -0.005188517, 0.00772455, -0.028258655, 0.031545147, 0.038997978, 0.042155083, -0.018243913, -0.0063594915, 0.0019731892, 0.002109048, 0.02673186, -0.04769295, 0.016704177, -0.02463575, -0.01594078, -0.01986128, 0.022552582, 3.211992e-05, -0.017467575, 0.0036358435, -0.0024988337, -0.03338248, -0.010816958, 0.0057351873, 0.0032848746, 0.018399179, 0.0102023585, -0.026524836, -0.020210631, 0.022099718, 0.02895736, 0.013534136, 0.0072522787, 0.008824361, 0.030095985, 0.0019812759, -0.03863569, -0.009173712, 0.036720727, -0.0075239963, 0.021168115, 0.010170011, -0.0071617058, -0.04316432, 0.0069611524, 0.04192218, -0.012913067, -0.020767007, -0.0121237915, -0.022863116, 0.014724519, 0.011127492, -0.007090542, -0.03421057, 0.011438027, -0.019706013, -0.0080997795, -0.0020330318, 0.0036229047, 0.009503655, -0.014608068, 0.006514759, 0.034365837, -0.014129328, 0.03201095, -0.013107151, 0.030432397, -0.017092345, 0.00040292682, -0.00084345747, -0.026938884, -0.021672733, 0.020547044, 0.00082809245, -0.0001457653, -0.00032347362, 0.011069267, 0.0023209231, 0.0063950736, 0.008151535, 0.03860981, -0.0036455477, 0.021349259, 0.005495817, -0.014569252, -0.027404685, -0.012900128, -0.012511959, -0.0073751984, -0.0052338033, -0.017920438, 0.014116389, -0.02686125, -0.025890829, 0.026990639, -0.0010011508, 0.023238344, 0.0077957143, 0.05998495, -0.0063336133, -0.02341949, -0.030949956, 0.00285142, 0.049866695, -0.007808653, 0.014012877, -0.009037853, -0.024351094, -0.0010027682, -0.015811391, 0.004033716, -0.026938884, -0.0021074307, -0.0100212125, -0.0017516096, 0.0003932226, -0.0020621442, -0.0017483749, 0.0034320552, -0.0018518865, -0.00309726, 0.03423645, 0.004269852, -0.03040652, -0.028025754, 0.03547859, -0.02758583, 0.004502753, -0.0036973036, 0.012065566, -2.07478e-05, 0.0061880504, -0.01692414, -0.019524869, 0.0003208454, 0.009374266, 0.040628288, 0.015177382, -0.024583995, -0.0021381606, 0.018243913, 0.012602532, -0.014129328, -0.032942552, 0.042465616, 0.018450934, 0.024868652, 0.0076145693, -0.011412149, 0.051212344, 0.008591459, 0.023548879, 0.008889055, -0.019227272, -0.017855745, -0.007892757, -0.03188156, 0.008267986, 0.010946347, 0.0026929178, -0.05791472, -0.038972102, -0.030354764, -0.0053696623, -0.015487917, -0.007879818, 0.018386241, -0.028672703, 0.01226612, 0.03198507, -0.010241175, -0.030613543, 0.016962957, 0.015707878, 0.009775373, -0.004256913, 0.011806787, 0.023924109, -0.009031383, 0.006967622, -0.038273398, 0.01630307, -0.043578368, -0.040033095, 0.019641317, 0.011043389, 0.0133141745, 0.012893658, -0.008669093, -0.023652392, 0.03726416, -0.026887128, 0.0008580138, 0.009147834, -0.022022083, -0.0046192035, -0.038221642, -0.0038590403, 0.008416784, 0.007064664, 0.043707754, -0.01018295, 0.05434357, 0.0076792636, 0.015099749, 0.0050009023, 0.010564649, -0.015720818, -0.027508197, -0.039903708, 0.029190259, -0.0022643153, 0.020676434, -0.0077310195, 0.009613636, -0.029216137, -0.03565973, -0.004243974, -0.007653386, 0.011237473, 0.004829461, -0.0050688316, -0.010247644, -0.003058443, -0.008526765, -0.0040984107, -0.0009890206, 0.002487512, -0.0020767006, -0.0009639514, 0.024066437, -0.018916737, -0.031622782, 0.021724489, 0.001722497, 0.00248266, -0.012971292, -0.012376101, 0.01189736, -0.010144133, 0.006058661, -0.043500733, 0.035607975, 0.011664459, 0.020029487, -0.0063789, 0.032657895, -0.002764082, 0.03472813, -0.02207384, 0.017558148, 0.02121987, -0.0051658736, -0.037186526, 0.036229044, -0.0078280615, -0.0024357564, -0.029604306, -0.015500856, -0.030613543, -0.012156139, 0.010157072, -0.004133993, 0.014388107, -0.013145968, -0.038972102, 0.0010634195, -0.011043389, 0.009613636, 0.0042116265, 0.015358527, -0.01751933, 0.011192188, -0.019136699, -0.00062753883, -0.005964854, -0.009134895, -0.02563205, 0.00030446955, 0.015410284, 0.040291876, 0.011806787, 0.023561819, 0.0010537154, -0.029888963, -0.005091475, 0.009257815, 0.011321576, -0.024739262, 0.014621007, 0.004402476, 0.0019440765, -0.008513826, 0.01741582, 0.006116886, -0.022449069, 0.04461348, 0.019279027, -0.02856919, 0.007931573, 0.004926503, -0.032269727, -0.009374266, 0.007187584, 0.046916615, -0.020922273, -0.01471158, 0.028724458, 0.0090119755, 0.026447203, 0.03591851, -0.01973189, 0.034546982, -0.018179217, -0.013534136, 0.0115286, 0.007808653, -0.0032492925, -0.012751331, 0.0030940252, 0.00039645736, -0.0005021928, 0.010435259, -0.006427421, 0.0109334085, 0.027404685, 0.00011604616, 0.009678331, 0.015216199, 0.067541294, 0.012725452, -0.011729154, 0.03348599, 0.021181053, -0.0145045575, 0.018218035, 0.018554447, 0.033641256, 0.011974993, 0.0019295202, 0.0021316912, -0.028051632, -0.0047000716, 0.03361538, -0.0035840878, -0.025541477, 0.008908464, -0.023949986, -0.044328824, 0.006650618, 0.005324376, 0.01004709, -0.011865012, 0.0055475724, 0.01704059, 0.0035873225, 0.0015615689, -0.03149339, -0.0043248422, 0.011987932, 0.025644988, -0.0002581724, -0.017286431, -0.022371436, -0.001059376, -0.03617729, 0.001704706, 0.0094713075, 0.004793879, 0.0072458093, 0.02120693, 0.0032120931, -0.045803864, -0.037626453, -0.03578912, -0.015682, 0.010409381, 0.028388046, 0.038325153, -0.022966627, -0.03571149, 0.006142764, -0.009794781, 0.0049523814, -0.004243974, 0.036461946, -0.031364, 0.0094713075, -0.013715281, 0.012427856, -0.02156922, 0.0044607013, 0.008889055, 0.0016755933], "content_hash": "a4af56a9da1686fd3a5b7bcead22b62d"}, {"content": "# MLP (Feed-Forward) Layers Explained\n\n## What Are MLP Layers?\n\nAfter attention gathers context from other tokens, each token's representation passes through a **Multi-Layer Perceptron (MLP)**, also called a **Feed-Forward Network (FFN)**. While attention handles relationships between tokens, the MLP processes each token independently -- and this is where much of the model's **factual knowledge** is stored.\n\n## What They Do\n\nThink of the MLP as the model's **memory bank**. During training, the MLP weights learned to encode facts, patterns, and associations from the training data. When the model processes \"The capital of France is,\" the MLP layers help recall that \"Paris\" is the answer.\n\nResearchers have found that specific facts are often stored in specific MLP neurons. This is one of the key findings in mechanistic interpretability research.\n\n## The Expand-Then-Compress Pattern\n\nEach MLP layer follows a distinctive pattern:\n\n1. **Expand**: The token's representation is projected into a much larger space (typically 4x the hidden dimension). For GPT-2, this means going from 768 dimensions to 3072 dimensions.\n2. **Activate**: A non-linear activation function is applied (like GELU or SiLU), which allows the network to represent complex patterns.\n3. **Compress**: The expanded representation is projected back down to the original size (768 for GPT-2).\n\n**Why expand then compress?** The expansion creates space for many individual neurons to each \"vote\" on whether a specific concept or fact is relevant. The compression then combines these votes into a refined representation. Each neuron in the expanded layer can activate for specific concepts.\n\n## Attention + MLP = One Layer\n\nIn each Transformer layer, attention and MLP work together:\n\n1. **Attention** gathers relevant context from other tokens\n2. **MLP** retrieves stored knowledge and transforms the representation\n3. The result is added back to the **residual stream** (the running representation)", "source_file": "mlp_layers_explained.md", "chunk_index": 0, "embedding": [-0.009905266, 0.003195755, -0.0029013676, -0.0052926736, 0.048033927, -0.010440515, 0.02612018, 0.04365118, -0.03924324, 0.090677574, 0.008123199, -0.011882541, -0.031283762, 0.003746747, -0.02047802, -0.049923044, 0.016410122, 0.006382064, 0.039293617, 0.021548519, -0.014306907, -0.023173159, 0.032820243, 0.031233387, 0.019420113, -0.017127987, -0.014634353, 0.029117577, -0.013400131, 0.031686775, 0.012694861, -0.022543453, -0.038336463, -0.019520868, 0.012776722, -0.00026132775, -0.018462962, 0.024382193, -0.013727577, 0.071030766, 0.014571383, -0.038059395, -0.056421604, 0.022908682, 0.025591226, 0.009584116, -0.037631195, 0.008620666, -0.0344071, 0.08513617, -0.020075008, 0.0453136, -0.02406734, 0.030956317, -0.040452275, 0.016976858, -0.015125523, -0.02112032, -0.011699926, 0.006375767, -0.0081924675, -0.0131608425, 0.039772194, 0.025792733, 0.010774259, 0.005119505, -0.07249168, 0.013992053, 0.00093826104, 0.04866363, 0.04375193, -0.007033809, -0.023185752, 0.034356724, 0.014798077, 0.007241612, -0.0007772926, 0.054658428, -0.0069141653, -0.037983827, 0.053953156, -0.028714566, -0.0044079376, -0.05138396, 0.031711962, 0.0034130032, -0.030805187, -0.008362488, -0.05309676, -0.06654727, 0.0024164945, 0.0029958233, -0.05027568, 0.03909211, 0.024822986, 0.022795334, -0.0072101266, 0.006334836, -0.011907729, 0.0027565353, 0.041686498, 0.002468445, -0.028387118, -0.044910587, 0.002139424, -0.052038852, 0.00899849, 0.0006096335, -0.032064598, -0.026170556, -0.067806676, -0.0031469527, 0.030150294, 0.030931128, 0.040175203, -0.012550028, -0.014722512, 0.043248165, 0.008274329, 0.0035106076, -0.03090594, -0.0213722, -0.0012586237, 0.012694861, 0.019357143, -0.027883355, 0.026220933, 0.010327169, 0.032165352, -0.00066591345, 0.005982201, 0.015289247, 0.027832977, -0.042416956, 0.007827238, -0.016636817, -0.02167446, 0.012984525, -0.023248723, 0.021989312, -0.0045118392, -0.028034484, 0.012764128, 0.02229157, -0.03543982, 0.011196162, -0.0043575615, 0.035666514, -0.04644707, -0.014306907, -0.004033263, -0.014722512, -0.05914193, -0.014886236, -0.040678967, -0.045565482, 0.024180686, 0.07485937, 0.06030059, -0.03856316, -0.011945511, 0.03246761, -0.014369877, 0.014029836, -0.021435171, 0.057076495, -0.014470629, -0.0017017788, -0.018727439, -0.0933979, -0.045112096, -0.020075008, -0.04929334, -0.015654476, 0.005226555, -0.02449554, -0.09188661, -0.054809555, -0.029621342, -0.046321128, -0.005012455, 0.00090835005, -0.047202718, -0.0030619425, -0.03317288, 0.0030965763, -0.024294034, 0.04690046, -0.001547501, -0.00015850866, -0.011290617, 0.046371505, 0.058688544, 0.006467074, -0.0025125246, -0.004212729, 0.07541352, -0.01876522, 0.030654058, 0.0065048565, 0.018324427, 0.057630636, 0.020301702, -0.009842295, 0.061761506, 0.031157823, 0.0024149201, -0.0126822665, -0.02259383, -0.034533042, 0.020415049, -0.033575892, 0.018072546, -0.012669672, -0.035590947, -0.038487595, -0.07672331, 0.044255696, 0.00082570117, 0.0235132, 0.033097316, 0.01790882, 0.007827238, 0.01906748, -0.073146574, 0.01327419, 0.045641046, -0.065438986, 0.00981081, -0.0064607775, 0.03201422, 0.0006226212, -0.0041119764, -0.06463296, -0.015377405, -0.05405391, -0.0038852822, -0.0035452414, 0.022241194, -0.013878707, -0.019306768, 0.049066644, -0.0021551666, 0.0073927413, 0.017833257, 0.010755368, -0.014722512, -0.0067126593, 0.05491031, 0.047353845, 0.06095548, 0.012984525, -0.055565204, 0.029848035, -0.022467889, -0.01045311, 0.01821108, 0.020276513, -0.04541435, -0.009244075, -0.003992332, 0.041862816, -0.028588625, -0.027883355, -0.02507487, -0.033575892, -0.028235989, -0.0114102615, -0.025490474, -0.029067202, 0.040427085, 0.0096785715, -0.016183428, 0.009269264, -0.023311693, -0.014684729, 0.025037086, 0.023676923, 0.012442978, 0.019697184, -0.03753044, -0.027681848, 0.081660196, 0.038613535, -0.02753072, -0.04148499, -0.00938261, -0.015553723, -0.0034224489, -0.028336743, 0.021687053, -0.026321685, 0.034331538, 0.005176178, 0.01171252, -0.0033846665, -0.0007871317, 0.021334419, 0.038840227, -0.0025518811, 0.0059160823, -0.0044362745, -0.010528674, -0.02363914, -0.018173298, 0.0058783, -0.025163027, 0.010843527, -0.0050344947, -0.0057492102, -0.022404918, -0.035112374, 0.032845434, -0.016183428, 0.016611628, 0.051585466, 0.011636956, -0.024999304, -0.0110387355, -0.0039419555, -0.008419161, -0.022845712, -0.004319779, -0.014848453, 0.022820523, 0.01045311, -0.035842832, 0.017581375, -0.010742774, -0.006769333, 0.03344995, -0.00803504, -0.023689516, 0.0020512652, -0.04506172, -0.0136016365, -0.037253372, 0.07581653, 0.015578912, 0.010320871, -0.029142765, -0.01671238, -0.030981505, -0.015037364, -0.06619463, 0.046623386, 0.027379591, -0.00525804, 0.013324566, -0.0020685822, -0.037177805, 0.00829322, 0.023865834, -0.030125106, -0.016599035, 0.033147693, 0.013866113, 0.00043095462, 0.0006108142, -0.008992193, -0.05025049, -0.027278837, 0.03430635, 0.0029234074, -0.04473427, -0.0010468852, 0.018966727, 0.035187937, -0.023387259, -0.004181244, 0.0041843923, 0.019697184, 0.045565482, 0.017921416, -0.0035043105, 0.0051478418, -0.033651456, -0.004234769, -0.01622121, -0.00506598, 0.0038286087, 0.04105679, 0.022921275, 0.007915397, 0.011951808, 0.03299656, 0.00882847, -0.030452553, -0.06977136, 0.01750581, 0.0100312065, -0.02556604, -0.014180966, 0.018085139, 0.007978368, -0.008941816, 0.005903488, 0.0054501, 0.049998607, 0.06292016, 0.01989869, -0.0053524957, -0.034784924, -0.013803142, 0.031938657, 0.027278837, 0.022606423, -0.010037504, 0.027127707, 0.038764663, 0.047958363, -0.017732503, 0.006224638, 0.0021095132, 0.0053871297, 0.032845434, 0.04035152, 0.012424087, 0.018034762, 0.009672275, 0.0136897955, 0.0014034559, -0.0037373016, -0.00052541046, -0.027782602, 0.028160425, 0.052139606, -0.06317204, -0.03299656, -0.012178502, -0.005831072, 0.026749885, 0.020037225, -0.017379869, 0.027429966, -0.030578494, 0.015276653, 0.05889005, -0.019697184, -0.041686498, 0.040754534, -0.028059673, -0.02274496, -0.056421604, -0.036623664, 0.017266521, -0.041988757, -0.06815931, -0.028865695, 0.00393251, 0.0017773435, 0.029848035, 0.038109772, 0.04486021, 0.012014778, -0.008305814, 0.06508635, 0.044809837, 0.07229018, -0.037681572, 0.029948788, -0.010812041, -0.007852426, -0.02535194, -0.018085139, -0.0031264871, 0.032719493, -0.036825173, 0.030326612, 0.013803142, 0.019999443, -0.021271449, 0.0042819967, -0.01897932, -0.002504653, 0.00807912, 0.0026463368, 0.020956594, 0.00998083, -0.06140887, 0.02406734, 0.017619157, -0.033701833, -0.026850637, -0.026749885, 0.07067813, -0.020528395, 0.01246187, 0.013714983, -0.012852287, -0.003979738, -0.016624222, 0.022354541, -0.013261596, 0.028361931, -0.047101963, -0.05420504, -0.06815931, 0.0072605032, -0.013488289, 0.007147156, -0.008532508, -0.009955642, 0.05224036, -0.019634213, 0.022430105, 0.025717167, 0.035792455, -0.018072546, 0.035666514, -0.015276653, 0.012342226, 0.0003160334, 0.02520081, -0.010264197, 0.015112929, 0.041736875, 0.022190819, 0.00981081, 0.009508551, 0.017102798, 0.020377267, -0.0344071, 0.019659402, 0.028336743, 0.029193142, -0.02528897, -0.030855564, -0.057630636, -0.028991636, 0.0077201882, -0.025603822, -0.0022275827, 0.006567827, -0.021926342, 0.004105679, -0.01919342, -0.04737903, -0.052139606, -0.026246121, 0.0037561927, 0.022153035, 0.02413031, 0.022644205, 0.0023629696, -0.016536063, 0.019117856, -0.041006416, 0.017329492, -0.0014003074, 0.00959671, -0.0064733713, 0.016019706, -0.02188856, -0.025087463, 0.0040553026, -0.017770287, -0.007934288, 0.011233944, -0.02541491, 0.0059727556, -0.0064198463, 0.0038286087, 0.0049809697, -0.042139884, 0.031686775, 0.010761665, -0.0029328528, 0.01947049, -0.04548992, 0.01579301, -0.007140859, 0.018777814, 0.0005671284, 0.0074809, -0.00082727545, 0.007386444, 0.01701464, -0.010106771, 0.00018281135, -0.031384517, -0.0127893165, -0.029747283, -0.0404019, -0.02130923, -0.008834766, -0.011731411, -0.029016824, -0.038487595, 0.0080161495, -0.011215053, 0.00042387043, 0.01186365, 0.017593969, -0.026372062, -0.003919916, 0.015830794, -0.017581375, -0.02336207, 0.008406567, -0.0055602985, 0.013186031, -0.010994656, 0.0004093085, 0.012191096, 0.032946184, -0.018840786, -0.0032366856, 0.007814644, -0.013349754, -0.013538666, -0.040678967, -0.004080491, -0.023462823, -0.033525515, -0.0028809023, -0.023979181, 0.0038034206, 0.014848453, 0.006630798, 0.006687471, 0.010037504, -0.0047290875, -0.009080351, 0.010037504, 0.014609165, -0.006410401, -0.011775491, -0.012027373, 0.021913748, -0.024545915, 0.0047574244, 0.044784646, 0.020654337, -0.007443118, 0.0059601613, 2.3220386e-05, -0.02639725, 0.02443257, -0.033122502, 0.000116495496, 0.0013192327, -0.024608886, 0.003913619, -0.007902803, 0.0072919885, 0.0042032837, 0.015654476, -0.01267597, -0.04065378, -0.0060325777, -0.02152333, 0.01897932, 0.013664607, -0.01771991, -0.010881309, -0.015578912, 0.0049211476, -0.028387118, 0.018790409, -0.03178753, -0.008545103, -0.015541129, -0.0457418, -0.020792872, 0.03669923, 0.015364812, 0.013412725, 0.032543175, 0.0051604356, -0.07289469, 0.019911284, -0.031938657, 0.040628593, 0.009495957, -0.05055275, -0.0037184104, -0.029142765, -0.0035043105, 0.032543175, 0.031157823, 0.015906358, -0.040578216, 0.0037687868, -0.00333429, -0.016384935, -0.028815318, -0.025742356, 0.024634074, -0.0049557816, -0.017883632, -0.017065017, 0.04206432, -0.0344071, -0.020024631, -0.0029958233, 0.013576448, 0.035087183, -0.01814811, -0.050653502, -0.0062718657, -0.010371247, -0.08024965, 0.010698695, -0.016976858, -0.031409703, -0.04543954, 0.061610375, 0.009325936, -0.05460805, -0.04400381, -0.001021697, 0.028487872, 0.022203412, 0.01023901, 0.0064072525, -0.012713752, -0.0066056093, -0.0061081424, -0.017984387, -0.0063064992, 0.0060861027, -0.030931128, 0.002599109, 0.036497723, -0.021321824, -0.05662311, -0.007940585, -0.01836221, 0.037782323, -0.005125802, 0.0047574244, 0.0070778886, -0.014042431, 0.00033984412, 0.0019442154, -0.009477066, -0.013198625, -0.055917837, -0.024231063, 0.005336753, -0.0074557117, -0.029167954, 0.013463101, 0.024243657, 0.039016545, -0.00012151346, 0.007329771, -0.008582884, -0.037681572, -0.029596154, 0.009615601, 0.0152640585, 0.002772278, -0.029319083, 0.010113069, 0.027329214, -0.016951669, 0.024684452, -0.0022212858, 0.0024778908, 0.022480482, -0.03516275, 0.012707455, 0.009634492, -0.04193838, 0.0062655685, -0.0060986965, 0.0215863, 0.009546334, 0.001726967, -0.00022197115, 0.01105133, 0.0038632425, 0.01784585, 0.0009406224, 0.040250767, -0.01160547, -0.008652152, -0.012707455, 0.0039419555, 0.0047763153, -0.0045748097, -0.03528869, -0.02075509, 0.014760294, -0.041258298, -0.026523191, -0.010075286, -0.037429687, -0.024016963, -0.0068323035, 0.025591226, -0.040250767, -0.007852426, 0.040301144, -0.0061207362, -0.09208812, -0.034810115, 0.008790688, -0.034079656, -0.015994517, -0.02387843, 0.03936918, 0.024961522, 0.005768101, -0.06548936, 0.0365481, -0.03531388, -0.01229185, -0.0032429828, -0.0005734255, -0.017770287, -0.01722874, 0.0075564645, -0.0009768305, -0.03261874, -0.034356724, -0.014596571, 0.01013196, -0.0027266243, -0.023261316, 0.011737708, -0.017606562, -0.014583977, -0.005396575, 0.014810671, -0.016057488, 0.023739893, -0.010629427, 0.01019493, 0.0048676226, 0.00677563, -0.0065111537, 0.01100725, 0.028941259, -0.0033500327, 0.002197672, -0.022417512, 0.0030572196, -0.04584255, 0.013941677, -0.024671856, 0.010213821, -0.034533042, 0.015100335, 0.02161149, -0.015654476, -0.03221573, 0.0008634835, -0.0016860361, 0.02118329, 0.008223953, -0.033827774, -0.0071912357, 0.0399737, 0.04742941, 0.0123296315, 0.04334892, 0.012361117, 0.0069960267, -0.0060199834, 0.0037026678, 0.03214016, 0.034507856, -0.0074179294, -0.028664188, 0.0038506484, -0.023097593, 0.031208199, -0.0043040365, 0.015427782, -0.020137979, -0.0018167001, 0.04682489, -0.018261457, 0.0020449683, 0.045137282, -0.018651873, -0.0060798055, 0.013085278, 0.036321405, 0.00013273009, 0.007115671, -0.05027568, 0.011548797, 0.050729066, 0.036774796, 0.018677061, -0.014105401, -0.0011193013, -0.0042410656, 0.0052391486, 0.013929083, 0.07098039, 0.04639669, -0.020591367, 0.066295385, 0.02486077, -0.0448854, -0.050653502, -0.056119345, 0.0047479784, 0.00639151, -0.00094140955, -0.008557696, 4.885136e-05, 0.02379027, -0.004011223, 0.023752486, 0.02090622, -0.011945511, 0.0070967795, 0.019457897, 0.018941538, -0.011888837, 0.01698945, -0.0024952076, -0.0077831587, 0.0075564645, 0.0023078702, -0.012209987, -0.036951113, -0.016460499, 0.055968214, 0.01181957, -0.02455851, -0.010912795, 0.013500883, -0.013929083, -0.0027738523, -0.01827405, 0.032114975, 0.005169881, 0.04148499, -0.013148248, 0.023336882, -0.009137025, -0.020994378, 0.0043764524, 0.017694721, -0.032291293, 0.019495679, 0.0039954805, -0.0019457897, -0.0017521553, 0.02109513, 0.014193559, 0.012839693, 0.003274468, 0.016775351, 0.0057460614, 0.013777954, -0.0156166935, 0.0008013001, 0.009042569, 0.037253372, -0.0008516765, 0.03624584, 0.015037364, -0.013727577, 0.024117716, 0.013349754, -0.0029564667, -0.008393973, -0.023840645, 0.004599998, -0.021775212, 0.004045857, 0.04276959, 0.020352079, 0.023286505, -0.005730319, -0.009124431, 0.041988757, 0.01083723, 0.029999165, -0.008790688, 0.04332373, 0.0123988995, 0.01263189, -0.0035893207, -0.009823404, -0.007367553, 0.019860908, -0.023374664, 0.006117588, -0.0014380896, 0.013840924, -0.01355126, -0.008022446, -0.00910554, 0.001021697, -0.01855112, -0.004011223, -0.015906358, -0.0036900735, 0.0011507866, 0.024835581, 0.006763036, -0.03299656, -0.023387259, 0.021195883, 1.9530706e-05, -0.0067063626, 0.0012916832, 0.039898135, 0.008022446, 0.005875151, -0.030553306, 0.015906358, -0.053046383, 0.00060727214, -0.0061585186, -0.008929223, 0.009401501, -0.015855981, -0.031837903, -0.047933172, 0.004930593, -0.016674599, -0.0060986965, 0.00048015037, -0.016636817, 0.012505949, 0.01147953, -0.019546054, -0.0084758345, 0.027933732, -0.002748664, 0.043147415, 0.008324705, 0.0049274447, 0.009779325, 0.0031737152, 0.0378327, -0.020893624, 0.020427642, 0.01855112, 0.010264197, -0.0046818596, -0.0077516735, -0.01126543, 0.0033594782, 0.006174261, -0.004200135, -0.031686775, -0.025943862, 0.0369763, -0.0022291571, -0.039192863, 0.0022039688, -0.0049274447, 0.026044615, 0.020276513, 0.00089654303, 0.03304694, 0.018865973, 0.017379869, 0.05289525, -0.04206432, 0.016687192, -0.01756878, -0.01989869, -0.010541269, 0.010169742, 0.015893765, -0.018160703, 0.041711684, -0.0020969189, 0.0077579706, 0.01622121, 0.017871039, 0.0002778575, -0.015918951, 0.012128126, -0.029822847, -0.0035641324, -0.013878707, 0.015125523, -0.009225184, 0.019294173, 0.030402176, -0.0027707038, -0.031409703, -0.008154685, -0.023916211, -0.007247909, -1.6160799e-05, 0.027631473, -0.0038191632, -0.001224777, 0.013941677, 0.02406734, 0.004634632, -0.01355126, -0.050200112, 0.029067202, -0.006637095, 0.0027738523, 0.014294312, -0.024268845, 0.014596571, -0.0065111537, -0.03924324, -0.0021315527, -0.008375081, -0.032291293, -0.006870086, 0.012606702, 0.028210802, 0.008456944, 0.017556187, 0.009823404, -0.07269319, 0.021699648, -0.028664188, 0.024797799, 0.04954522, 0.0011610193, 0.022090064, -0.00040812782, -0.011687332, 0.016674599, -0.0042379173, -0.0033720722, 0.018198486, 0.01105133, 0.011183567, -0.008948114, 0.016762758, -0.038789853, -0.013022307, -0.0057806955, 0.0017316898, -0.006901571, -0.029570965, 0.027253648, -0.008872549, 0.0048928107, -0.0049148505, 0.0061207362, 0.019243797, -0.0058908937, -0.018840786, 0.019432709, 0.0006430866, 0.05067869, -0.032517985, 0.0017017788, -0.00412457, 0.005957013, 0.040981226, -0.041837625, 0.024168093, -0.037026677, -0.031686775, -0.003759341, 0.0222286, -0.0006112078, -0.03672442, 0.012688563, 0.011744006, -0.026019426, 0.004011223, 0.01115838, 0.034054466, -0.014029836, 0.009817107, 0.0017175215, 0.001171252, -0.0076446235, 0.025125245, -0.0029895264, 0.0012901089, 0.004477205, -0.041711684, 0.02152333, -0.047202718, 0.0037750837, 0.039218053, 0.0029911005, 0.029394647, 0.02066693, -0.008274329, -0.040880475, 0.030729624, -0.006637095, -0.019583838, -0.005138396, -0.0024149201, -0.033399574, -0.0073045823, -0.023828052, -0.030402176, -0.013463101, 0.009968236, 0.02336207, -0.004099382, 0.0024652965, -0.01079315, 0.018034762, -0.044406824, -0.012625593, 0.032089785, -0.006681174, -0.007915397, -0.0050596828, 0.026346873, 0.0050156033, 0.0050691282, -0.006630798, -0.034633797, -0.0048361374, 0.02556604, 0.006542639, 0.024105122, 0.00272505, -0.023689516, -0.007997259, 0.007033809, -0.0018513339, 0.017556187, 0.015969329, -0.00046598198, 0.01658644, -0.0015530109, -0.04148499, -0.007115671, -0.0062183407, -0.0002560146, 0.0056075263, -0.026749885, 0.021913748, -0.038311277, -0.028160425, 0.04176206, 0.0030524968, 0.014193559, 0.019671997, 0.022430105, 0.018803002, -0.0069582444, -0.025704574, -0.013173437, 0.0021756322, -0.00068677246, 0.003327993, -0.013639418, -0.016208617, 0.006574124, -0.022493076, 0.0030210116, -0.041434616, -0.006769333, 0.024193281, 0.027757414, -0.0031091704, -0.023034623, 0.026447626, -0.019495679, 0.02498671, 0.014382471, 0.00033708918, 0.05632085, 0.0004726726, -0.03994851, 0.012959337, -0.03712743, 0.033248443, -0.010931686, 0.0115425, 0.012310741, 0.019483084, -0.006095548, -0.034079656, 0.0075123855, 0.0054469514, 0.042845156, 0.035616137, -0.041107167, -0.0016954817, -0.011347291, 0.0045685126, 0.010963171, -0.011397667, 0.011278024, 0.008608073, 0.004417383, 0.01115838, 0.009458175, 0.055414073, 0.028739754, 0.020780278, 0.013576448, -0.030452553, 0.023273911, -0.016271587, -0.02066693, 0.004420532, 0.016145647, 0.014231342, -0.031308953, -0.020150572, -0.015138118, 0.0064324406, -0.0294954, 0.035842832, 0.003894728, 0.0048928107, 0.005097465, 0.010774259, 0.0018182743, -0.0026636538, 0.022417512, 0.014168371, -0.005418615, -0.016145647, -0.026145367, -0.028764943, 0.015704852, -0.014810671, -0.02011279, 0.013400131, 0.0020717308, -0.01622121, 0.00916851, 0.014495818, -0.008494725, -0.02293387, -0.02753072, -0.007134562, 0.0029863778, -0.011296915, 0.011133191, -0.0023346327, -0.0054626944, 0.0014845304, -0.029142765, -0.021926342, 0.002235454, 0.0033028047, 0.047907986, -0.016170835, 0.04629594, 0.06251715, -0.0001902891, -0.0012586237, -0.00037979108, -0.036271032, -0.04105679, -0.018601498, 0.03347514, -0.0007182577, -0.0020937703, 0.011788085, -0.00026467306, -0.010358654, 0.0059066364, -0.026498003, -0.03221573, 0.016233806, 0.0064198463, 0.010919091, 0.004817246, -0.015150712, -0.007619435, -0.008098011, 0.0016624222, 0.015868576, -0.022241194, -0.0067504416, -0.0060231322, -0.017682128, -0.022064878, 0.0127011575, -0.027379591, 0.0071786414, -0.024306629, 0.00835619, 0.015969329, 0.02139739, 0.011599173, 0.02096919, -0.019495679, 0.012801911, 0.013639418, 0.021296637, 0.056975745, 0.01545297, 0.015755229, -0.011271726, 0.017392462, -0.0013507181, 0.022430105, -0.062315647, 0.017127987, 0.01100725, 0.006035726, 0.00081940414, -0.0119707, -0.014306907, -0.016561251, 0.007732782, 0.00463778, 0.027883355, -0.005610675, -0.027329214, 0.0003849074, -0.008110605, -0.007990961, -0.028387118, 0.024722233, -0.0048865136, -0.017530998, -0.0072605032, 0.020843249, -0.012971931, -0.025616415, -0.0016057488, -0.0119707, -0.009495957, -0.00058641314, -0.019331956, 0.00033374387, 0.03642216, -0.0149618, -0.025339345, -0.0060231322, -0.0058594085, -0.01855112, 0.03105707, -0.016838321, 0.025817921, 0.00097132055, -0.0059790527, -0.023374664, 0.0050030095, 0.017266521, -0.006328539, -0.028664188, 0.008960708, -0.028437495, -0.020062413, -0.04065378, 0.00987378, 0.049923044, -0.016951669, 0.018311832, 0.025717167, 0.039721817, 0.029167954, 0.042517707, -0.04332373, 0.04586774, -0.0031752894, -0.027077332, 0.004256808, -0.014269124, -0.015188494, 0.0034980134, 0.009092946, -0.0218004, 0.0011311083, -0.025704574, 0.01051608, 0.031233387, 0.036019146, -0.0111268945, 0.0009579393, -0.014986988, 0.016674599, -0.021485548, 0.0316364, 0.0079091, 0.028840506, -0.020188354, 0.0057869926, 0.02342504, 0.031989034, 0.023475416, 0.015843388, 0.040124826, -0.004367007, 0.032366857, 0.027858166, -0.020276513, -0.06392769, -0.0102075245, -0.025603822, 0.017253928, -0.0077579706, -0.00899849, -0.036648855, -0.025213404, 0.0031359328, 0.014722512, 0.0098359985, 0.027203273, -0.0431726, -0.020452831, -0.016888699, 0.012317037, 0.00032449505, -0.001069712, -0.02413031, -0.006574124, -0.029570965, -0.0049683754, 0.02259383, -0.00040497928, 0.022417512, -0.0035357957, -0.011032439, -0.06125774, -0.018110327, 0.0070841857, -0.015150712, 0.02596905, 0.042013943, 0.031082258, -0.024835581, 0.0033972606, -0.025553444, -0.01673757, 0.02081806, -0.013614231, -0.002192949, -0.026623944, -0.019206014, -0.018865973, 0.026875826, 0.008286923, -0.014395066, -0.0049998607, 0.04203913], "content_hash": "904327181f4c28aa82a6a95107903e6a"}, {"content": "activate for specific concepts. ## Attention + MLP = One Layer In each Transformer layer, attention and MLP work together: 1. **Attention** gathers relevant context from other tokens 2. **MLP** retrieves stored knowledge and transforms the representation 3. The result is added back to the **residual stream** (the running representation)\n\nThis happens in every layer. GPT-2 has 12 such layers; each one further refines the model's understanding.\n\n## What You See in the Dashboard\n\nIn **Stage 4 (MLP/Feed-Forward)** of the pipeline, you can see:\n\n- The expand-compress flow: Input dimension \u2192 Expanded dimension \u2192 Output dimension\n- The number of layers in the model\n- An explanation of why the expansion matters for knowledge storage", "source_file": "mlp_layers_explained.md", "chunk_index": 1, "embedding": [-0.017636668, 0.022921301, 0.02181344, -0.011632561, 0.036750488, -0.002424246, -0.005262349, 0.05251525, -0.012606716, 0.07762681, 0.037998427, 0.0013935834, -0.04413624, -0.024538526, 0.012664019, -0.035145998, -0.012867765, 0.016490603, 0.021928046, 0.04051977, -0.009181255, -0.04497669, 0.04021415, 0.053126484, 0.012956902, 0.013650909, 0.0066280765, 0.045129497, 0.010779379, 0.022042653, 0.0063383766, -0.03927183, -0.051623866, -0.008041557, 0.028142266, -0.022590216, -0.018438915, 0.05577517, -0.00985616, 0.06652271, 0.011759901, -0.014007462, -0.054196145, 0.046886798, 0.008264403, 0.017814945, -0.0504778, 0.0011030877, -0.03792202, 0.06677739, -0.0057812617, 0.028575223, -0.022704823, 0.008347174, -0.023507068, 0.011868141, -0.015204464, -0.02558272, -0.005211413, 0.0058099134, -0.005969089, -0.0031055182, 0.039348233, 0.017942285, -0.012211961, -0.0056762057, -0.070801355, 0.03687783, -0.01836251, 0.038737, 0.01823517, 0.008296238, -0.042913772, 0.034916785, 0.022577483, 0.020743778, -0.011893609, 0.0440089, -0.023239654, -0.012886865, 0.036775954, -0.05979913, -0.00033048858, -0.059442576, 0.006147366, -0.014529559, -0.011957279, -0.008882005, -0.037947487, -0.061327215, 0.02018348, 0.014058399, -0.03743813, 0.01896101, 0.00029527093, 0.027327286, 0.025875604, -0.0051604765, -0.032217164, 0.05210776, 0.041945983, -0.004183138, -0.0014079091, -0.04347407, 0.00045763017, -0.051139973, -0.0093786325, 0.01613405, -0.03201342, -0.04563886, -0.0850635, -0.02005614, 0.0246404, 0.01848985, 0.040163215, -0.010021702, -0.021074863, 0.0049694655, 0.0013418512, 0.009308595, -0.029492075, -0.00981159, -0.003399993, 0.0077486737, -0.016210454, 0.02253928, 0.032166228, 0.01626139, 0.03580817, -0.002548403, -0.01829884, 0.019152021, 0.064332455, -0.046530243, 0.004905795, 0.004737069, -0.018757265, -0.00022264702, -0.026817923, 0.009607846, 0.019877862, -0.03211529, 0.033770718, 0.033694316, -0.04385609, 0.006793619, -0.029441139, 0.04861863, -0.034509294, 0.007634067, -0.016375996, -0.0003748588, -0.026359497, -0.03351604, -0.04765084, -0.026614178, -0.004062164, 0.06845829, 0.053686783, -0.04156396, -0.0073793856, 0.025939273, 0.0034127273, 0.004889878, -0.046632115, 0.06458714, -0.026435902, -0.013383494, -0.03685236, -0.09387547, -0.02395276, -0.006357478, -0.06438339, -0.024029166, 0.027174477, -0.005017218, -0.08241481, -0.04818567, -0.023366993, -0.038533255, -0.016363263, -0.011218704, -0.03858419, 0.0002011583, -0.040978193, 0.017369254, -0.0026248074, 0.033465102, 0.015701093, 0.015471878, -0.0020151644, 0.03779468, 0.0346621, 0.02273029, 0.004351864, -0.022615684, 0.062447812, -0.005392873, 0.02238647, 0.0057908124, 0.02297224, 0.04276096, 0.039297298, -0.04411077, 0.044339985, 0.039602917, -0.012409338, 0.0087992335, -0.03010331, -0.017827678, 0.02335426, -0.03914449, 0.016121315, 0.007182008, -0.019508574, -0.030918289, -0.08822154, 0.02011981, 0.0068318215, 0.0021170368, 0.026843391, 0.010511964, 0.007831445, 0.021074863, -0.06677739, 0.026537774, 0.043219388, -0.05358491, 0.009136686, -7.846865e-06, 0.030740013, 0.020667374, -0.011861774, -0.02043816, 0.004803923, -0.056793895, 0.016541539, -0.02517523, 0.024220176, -0.032319035, 0.00012087405, 0.03927183, -0.02281943, 0.01062657, 0.026410434, 0.039984938, -0.012059152, -0.008423579, 0.0428119, 0.024691336, 0.029721288, -0.004100366, -0.06412871, 0.004791189, -0.018528052, 0.009480505, 0.022361003, 0.0070737684, -0.032879334, -0.012734056, 0.03619019, 0.031274844, -0.04360141, -0.023443399, -0.015802965, -0.018795468, -0.016834423, -0.0041704034, -0.0044410024, -0.034229144, 0.023405196, 0.009499607, 0.0054024234, 0.01857899, -0.027658371, 9.737077e-06, 0.031682335, 0.03779468, 0.01600671, 0.04754897, -0.0022364187, -0.039042618, 0.0610216, 0.047472563, -0.020731045, -0.012071886, 0.0046606646, -0.0042213397, -0.008544552, -0.032777462, 0.04184411, 0.0041385684, 0.044212643, -0.017038168, 0.026334029, -0.0032233081, -0.013727313, 0.0044346354, 0.038966212, 0.0087992335, -0.002123404, -0.020743778, -0.014593229, -0.030740013, -0.0013689111, 0.036903296, -0.0033331392, 0.011231438, 0.012059152, -0.0077232053, 0.0078250775, -0.02564639, 0.0504778, 0.006596241, 0.036674082, 0.01990333, 0.020107076, -0.0091939885, -0.009143053, -0.015586485, -0.013358026, -0.02577373, -0.011562523, -0.05251525, 0.020221682, -0.016210454, -0.03494225, 0.022513812, 0.0066535445, -0.005523397, 0.041513026, -0.02821867, -0.041385684, 0.009034813, -0.03766734, 0.008671893, -0.011689864, 0.044365454, -0.034509294, 0.012574881, -0.035731763, -0.01675802, -0.05327929, -0.010110841, -0.05282087, 0.04497669, 0.028906308, 0.0016283675, 0.050859824, 0.0016379181, -0.0094041005, -0.036775954, 0.0038234005, 0.0007835424, 0.010244548, 0.038635127, 0.026690584, -0.0071438057, -0.0003760526, 0.0010911495, -0.059595384, -0.0076913703, 0.037743744, 0.030740013, -0.034891315, 0.04130928, 0.002175932, -0.0026184404, -0.007933318, -0.017942285, 0.03371978, -0.0045046727, 0.025952008, 0.017165508, -0.036826894, -0.0015153527, -0.01566289, -0.008684627, -0.025366241, 0.0046383804, -0.0017175059, 0.020692842, -0.01343443, 0.029619416, 0.009111217, 0.015102591, -0.008245301, -0.024372984, -0.064281516, 0.027098073, 0.018196966, -0.031758737, -0.020769246, -0.0018177866, 0.01962318, -0.009875261, 0.021304077, -0.0019610447, 0.066217095, 0.041411154, -0.025659123, -0.010620203, -0.046784922, -0.031071099, 0.023048643, 0.016121315, 0.029339267, -0.018247902, 0.03494225, 0.015408209, 0.055316743, -0.016426934, 0.0038170335, -0.0005869604, -0.009232191, 0.012797727, 0.02297224, 0.035400677, 0.023366993, 0.035935506, -0.011982747, 0.030485332, -0.014771506, -0.0040653474, -0.029772224, 0.026512306, 0.060410365, -0.04668305, -0.019992469, 0.0075449287, -0.012861397, 0.058474787, 0.0068636565, -0.01628686, 0.029110054, -0.024627665, 0.01356177, 0.021609694, -0.016044911, 0.00510954, 0.049026117, -0.029059118, -0.0073793856, -0.066166155, -0.052413378, 0.006978263, -0.040316023, -0.05979913, -0.027607435, -0.012543046, -0.006456167, 0.040341493, 0.019610448, 0.026716052, -0.051063567, -0.052566186, 0.04793099, 0.03254825, 0.06458714, -0.0061314483, 0.03468757, 0.0042213397, -0.014682367, -0.020883853, -0.02219546, 0.013536302, 0.013141546, -0.02740369, 0.01776401, 0.0055870675, 0.016184986, -0.032344505, 0.026232157, 0.00020871915, -0.0054851947, -0.001671345, 0.009219457, -0.010053538, 0.0063001746, -0.047166947, 0.0152554, 0.033465102, -0.018795468, -0.0017748092, -0.0093786325, 0.118987024, -0.045180432, -0.0037374457, 0.034916785, -0.020641906, -0.021329544, -0.014173005, 0.0111550335, -0.013956526, 0.030332522, -0.03428008, -0.052566186, -0.06198939, 0.001742974, 0.0015336579, 0.008678259, 0.008761031, -0.019202957, 0.049051587, 0.02545538, -0.0022857632, 0.041869577, 0.032344505, -0.00018862322, 0.020196214, -0.04209879, -0.006475268, 0.007996988, 0.009015712, -0.018948276, 0.008945675, 0.04591901, 0.022131791, 0.0057812617, 0.027505562, -0.006029576, 0.01619772, -0.021673365, 0.025009688, 0.006806353, -0.008187998, -0.021762503, -0.054858316, -0.06005381, -0.003559169, -0.005555232, -0.044645604, 0.013268887, -0.00047553744, -0.042735495, -0.00874193, -0.012307466, -0.032471843, -0.048924245, -0.00668538, 0.0066408105, 0.027047137, 0.029492075, 0.009932565, 0.0060072914, -0.013663643, 0.019292096, -0.013052408, 0.02599021, 0.038125765, 0.019928798, -0.009938931, 0.012415705, -0.014122069, -0.030841885, -0.003801116, -0.0070801354, -0.016936295, -0.0013291172, -0.014682367, 0.0007127092, 0.007977886, 0.012625817, -0.04413624, -0.023659877, 0.05424708, 0.010244548, -0.0038106665, -0.02376175, -0.034229144, -0.008047923, 0.0018878239, 0.0039284565, 0.02740369, 0.0016267757, -0.008347174, 0.015191729, 0.016363263, -0.006558039, -0.032140758, -0.019877862, -0.039195426, -0.025888337, -0.020960256, -0.013230685, 0.004310478, -0.0131033445, -0.028116798, -0.024895081, 0.0026168486, 0.003597371, -0.012644919, 0.014071133, 0.023099579, -0.023277856, -0.011116832, 0.01754753, -0.0047625373, -0.027913053, -0.015535549, 0.010410091, -0.016847158, -0.020234415, 0.004370965, -0.0031660048, 0.034763973, -0.038380448, 0.015981242, 0.0077677746, -0.014134803, -0.0029208742, -0.04133475, -0.019508574, -0.023277856, -0.057354193, -0.011715332, -0.020425428, 0.013447164, 0.029746756, -0.015089857, -0.012040051, 0.015357273, -0.002548403, -0.023023175, 0.013625441, 0.020399958, -0.028269606, -0.011791737, -0.032166228, 0.024003698, -0.029237393, 0.0034031766, 0.060104746, 0.010645672, -0.006526204, 0.018005956, -0.0061951187, -0.017305583, 0.023940027, -0.037310787, 0.0073666517, -0.006672646, -0.015102591, -0.00272668, 0.016146785, 0.008105227, -0.011046794, 0.015586485, -0.0047529866, -0.02686886, -0.034763973, -0.009677883, 0.016732551, 0.006309725, -0.0105629, -0.009034813, -0.020922055, 0.005249615, -0.0040112277, 0.0127085885, -0.038507786, -0.027709307, -0.015981242, -0.064943686, -0.023341525, 0.053432103, 0.031071099, 0.0123202, 0.025939273, 0.00369606, -0.05939164, 0.03970479, -0.0170127, 0.03145312, 0.019814191, -0.031478588, -0.0048707766, -0.02530257, -0.016821688, 0.02153329, 0.006147366, 0.018375244, -0.046810392, 0.019877862, 0.0075576627, 0.0053833225, -0.009525075, -0.032599185, 0.008512717, 0.019928798, -0.038635127, -0.013319823, 0.016299592, -0.037107043, -0.033974465, -0.0018655394, 0.007977886, 0.04469654, 0.012281998, -0.0142621435, 0.0072711464, -0.003559169, -0.05414521, 0.010620203, 0.0032328586, -0.032140758, -0.03035799, 0.06112347, 0.021164002, -0.04737069, -0.032777462, 0.018591722, 0.012352035, 0.0023955945, -0.019101085, 0.025162496, -0.01905015, -0.026970733, -0.010811214, -0.0077868756, 0.00024194708, 0.0041513024, -0.029364735, 0.011256906, 0.014313079, -0.013752781, -0.04411077, -0.011104098, -0.012810461, 0.029084586, 0.0042404407, 0.028575223, 0.0077041043, -0.03333776, 0.017814945, 0.012141923, 0.0066408105, -0.0060709617, -0.041360214, -0.017865881, -0.012154657, -0.015013453, -0.03415274, 0.029186457, 0.016694348, 0.016235922, -0.0010457844, 0.0058099134, -0.0018655394, -0.038304042, -0.025710061, 0.009187622, 0.00674905, -0.003960292, -0.022602951, 0.0036451237, 0.021966247, 0.00080622494, 0.033796187, -0.009270393, -0.009219457, 0.005437442, -0.018260637, 0.0046574813, 0.013039674, -0.033083078, 0.022093588, -0.00087864994, 0.028014924, 0.0211258, 0.001310016, -0.006927327, 0.035578955, -0.010677506, 0.017496593, 0.0049790163, 0.0317078, -0.034509294, 0.018553521, -0.0039157225, -0.00041863212, 0.010308219, -0.00522733, -0.033363227, 0.0008094085, -0.0006076533, -0.023430664, -0.014402218, 0.004921713, -0.023621675, -0.02860069, -0.0014660083, 0.026537774, -0.03858419, -0.02297224, 0.02024715, 0.018069627, -0.07182008, -0.013689111, 0.002661418, -0.008124328, -0.0043009277, -0.01090672, 0.029110054, 0.007398487, 0.0075258273, -0.056386404, 0.01685989, -0.014580495, -0.007818711, 0.011568891, -0.0013211584, -0.04360141, 0.011823571, 0.005294184, -0.00086830347, -0.032420907, -0.020107076, -0.031784207, 0.015955772, 0.02727635, -0.06290624, -0.01086215, -0.0068381885, -0.008869271, 0.0059277033, 0.012784993, -0.020883853, 0.02071831, -0.020196214, 0.024614932, 0.010620203, -0.0020422244, -0.0049853832, -0.020960256, 0.009397734, 0.0146696335, -0.009556909, -0.014007462, 0.011957279, -0.01214829, 0.00722021, 0.0032201246, 0.015484613, -0.035044122, 0.012071886, 0.026945263, -0.011829939, -0.026817923, -0.009951665, -0.0065453053, 0.026155753, 0.015242666, -0.019419437, 0.0036737754, 0.03035799, 0.011696231, 0.021978982, 0.03361791, 0.02234827, 0.01789135, 0.0030195632, 0.001953086, 0.025429912, 0.040825386, -0.010270016, -0.036801424, -0.0068572895, -0.031122034, 0.015522815, -0.007099237, 0.017687604, 0.0012726098, -0.005459727, 0.031376716, -0.0011572074, 0.015051655, 0.026359497, 0.01144155, -0.03117297, 0.008487249, 0.03104563, 0.0015694725, -0.009849793, -0.03496772, 0.01795502, 0.013867388, 0.039297298, -0.013943792, -0.008690993, 0.0022778043, -0.016172253, 0.026614178, 0.01795502, 0.04546058, 0.045282304, -0.017700339, 0.07136165, 0.013587238, -0.036088318, -0.04306658, -0.04795646, -0.0023940026, 0.00040052587, -0.021023927, -0.0047338856, -0.0028874474, 0.013740047, -0.022335535, 0.02266662, 0.029211925, -0.019725055, 0.026894327, 0.032981206, 0.029339267, -0.0015631055, -0.020323554, 0.0060391263, -0.0076977373, -0.026486838, 0.017229179, -0.0103527885, -0.0070355665, -0.0034763974, 0.044263583, 0.009238558, -0.023799952, -0.045409646, 0.018209701, -0.01986513, 0.009225824, -0.033796187, 0.034814913, 0.011772635, 0.017178243, -0.020896588, 0.00457471, -0.0032105742, 0.0058130966, 0.006729949, -0.009512341, 0.0036705919, 0.018922808, -0.0031007428, -0.024041899, -0.00452059, 0.026181221, 0.018184233, 0.025073357, -0.008156164, 0.028906308, 0.017878616, 0.020030672, -0.0023876356, -0.016350528, -0.003237634, 0.03160593, 0.030867353, 0.037107043, 0.015089857, -0.008245301, -0.0099580325, -0.0045174067, -0.005825831, 0.021647897, -0.028677097, 0.009327697, -0.038915277, 0.014007462, 0.02806586, 0.034509294, 0.026970733, 0.0041576694, -0.0134853665, 0.04360141, -0.0059117856, 0.009607846, -0.02316325, 0.06209126, 0.006997364, 0.0007942868, 0.014937049, -0.014542293, 0.023265122, 0.027581967, -0.032879334, 0.021584226, -0.02530257, 0.0061155306, 0.005322836, 0.0072584124, -0.019967001, -0.026015678, 0.0037469962, -0.0008042353, -0.02329059, 0.00060844916, -0.003092784, 0.024322048, 0.027301818, -0.018515319, -0.015624688, 0.022997707, 0.0078250775, -0.0025786464, 0.0023446581, 0.021775236, -0.0021743402, 0.015561017, -0.005975456, 0.023749016, -0.04171677, 0.0053037344, 0.019075617, -0.027352754, 0.037081573, -0.008684627, -0.046937734, -0.049127992, -0.021240406, -0.02273029, -0.0007823486, 0.013549036, -0.006774518, 0.008258035, 0.0022714373, -0.0035623524, -0.0048548593, 0.01424941, -0.009314963, 0.030714545, 0.012390237, 0.0029447507, 0.0062492383, -0.010416458, 0.04153849, -0.022220928, 0.012829562, 0.010295485, -0.0099707665, 0.010849416, -0.015306336, -0.0036833258, -0.0010999042, 0.019381234, -0.015637422, -0.03616472, -0.027098073, 0.033847123, 4.6807607e-05, -0.039883066, 0.01754753, -0.0042690923, 0.034356486, 0.012154657, -0.015535549, 0.008627323, 0.027327286, 0.03331229, 0.052311506, -0.056386404, 0.0072775134, 0.0027632902, -0.008939308, -0.012071886, 0.035655357, 0.022220928, -0.020743778, 0.029364735, -0.015828433, -0.0077550407, 0.013510834, 0.008964776, 0.014504091, -0.021775236, 0.0072966143, -0.026817923, -0.0009972358, -0.029313799, 0.0034827646, -0.027632903, -0.0004305703, 0.020412693, -0.026206689, -0.0020820182, 0.008480881, -0.035477083, -0.021482354, 0.024500325, 0.024551261, -0.008920207, -0.008353542, 0.02009434, -0.011823571, 0.005039503, 0.0015861859, -0.039984938, 0.017649403, -0.021928046, -0.0045778933, -0.0033777086, -0.0022809878, -0.0049821995, -0.005749427, -0.048567694, -0.006755417, 0.0010051946, -0.035680827, -0.025875604, 0.010161777, 0.025544519, 0.0056825727, 0.013007839, 0.020132544, -0.047599904, 0.010371889, -0.03687783, 0.012880499, 0.046377435, -0.01424941, 0.0075576627, -0.006570773, -0.020005204, 0.028040392, -0.009015712, 0.0010823949, 0.03331229, 0.0076850033, 0.028371477, -0.02234827, 0.02444939, -0.027047137, -0.030969227, -0.015879368, -0.0057685277, -0.00598819, -0.0037883818, 0.028040392, -0.0011842672, -0.0113396775, -0.001317179, 0.016528806, 0.0095887445, -0.011174135, -0.02997597, 0.027480094, -0.014058399, 0.039475575, -0.022781227, -0.0042499914, -0.01613405, -0.010524698, 0.036241125, -0.007659535, 0.0006649565, -0.05220963, -0.034636635, 0.022093588, 0.03751453, 0.0052591655, -0.03560442, 0.006736316, 0.019814191, -0.039781194, 0.0062428713, 0.031784207, 0.005555232, 0.013956526, -0.0034127273, -0.0008253261, -0.0075385617, 0.0057175914, 0.016159518, 0.007169274, -0.011829939, 0.020947523, -0.019546777, -0.011683497, -0.05740513, 0.011836305, 0.033006676, -0.00698463, 0.019483106, 0.0138801215, 0.01789135, -0.019928798, 0.005552049, 0.0026359498, -0.025035156, -0.00020951503, -0.0126958545, -0.026079347, -0.0035273337, -0.0078378115, -0.027378222, -0.017203711, -0.014453154, 0.0038393182, 0.002647092, 0.009486872, -0.0021616062, 0.029823162, -0.059748195, -0.02630856, 0.038915277, -0.014338548, 0.0103527885, -0.03952651, 0.022513812, 0.021482354, 0.024181973, -0.02090932, -0.023787217, -0.006140999, 0.009887995, -0.009817958, 0.020883853, 0.021838907, -0.007430322, -0.0032917536, -0.010792113, -0.0047784545, 0.012173758, 0.03252278, 0.005615719, 0.007735939, 0.016210454, -0.03782015, -0.014173005, -0.007850546, -0.01150522, -0.0035432512, -0.021953514, 0.041513026, -0.037107043, -0.041640364, 0.029899566, 0.011282374, 0.020196214, 0.004291377, 0.037845615, 0.035477083, -0.008245301, -0.048160203, -0.024971485, 0.007436689, 0.003820217, 0.0029145072, -0.0142876115, -0.014134803, 0.012020949, -0.024971485, -0.00739212, -0.028626159, -0.010346421, -0.007812344, 0.0021966249, -0.022806695, -0.009111217, 0.006526204, -0.015102591, 0.032853868, 0.014083867, 3.6635283e-05, 0.077474, -0.0134853665, -0.030740013, 0.026104815, -0.01905015, 0.026015678, 0.0015225157, 0.0077614076, 0.02024715, 0.0077805086, 0.00017877422, -0.046835862, 0.00066137506, -0.014020196, 0.048924245, 0.025366241, -0.048287544, 0.008283504, -0.015866635, 0.0009845018, -0.005115907, -0.005224147, 0.033821654, 0.01870633, 0.006367028, 0.021138534, -0.007659535, 0.03198795, 0.049026117, 0.014338548, -0.008519084, -0.041003663, 0.00081816316, -0.019279363, -0.032344505, 0.013307089, 0.017993223, 0.00492808, -0.039068084, -0.018260637, 0.0008014497, 0.007003731, -0.013039674, 0.026792455, 0.0142621435, -0.022602951, 0.005348304, -0.00066814, -0.004078082, -0.01131421, 0.013358026, 0.013141546, -0.0073220828, -0.0038584191, -0.01424941, -0.012441173, 0.013574504, 0.017662136, -0.014822442, 0.0099580325, 0.016656145, -0.016490603, 0.0001540025, 0.019559512, 0.009996234, -0.019725055, -0.032319035, 0.000655406, 0.02024715, -0.02686886, 0.013268887, 0.00096221716, 0.00043216205, -0.00657714, -0.02234827, -0.010333687, -0.0078059766, -0.011855407, 0.043779686, -0.0070865024, 0.055927977, 0.047192413, -0.013714579, 0.00043813113, -0.02354527, -0.043779686, -0.038686063, -0.032166228, 0.04548605, 0.0029479342, -0.006198302, 0.0099707665, -6.9838345e-05, -0.035986446, -0.0057112244, -0.036342997, -0.05195495, 0.005230514, 0.0047689043, 0.01133331, -0.0043295794, -0.0032010234, -0.020680107, 0.0003388453, -0.027429158, 0.034636635, -0.03236997, 0.00839811, -0.006672646, -0.017050901, -0.021698833, 0.013981994, -0.017725807, 0.0036833258, -0.02793852, 0.015535549, 0.0024528976, 0.003708794, 0.0017063636, -0.011753534, -0.0036196555, -0.00013211583, 0.01632506, 0.00897751, 0.038431384, 0.018413447, 0.00874193, 0.0063160923, 0.021966247, -0.013141546, -0.004160853, -0.05669202, 0.047498032, 0.012517578, 0.0032121658, 0.00089456746, -0.014338548, -0.03229357, 0.0001224658, -0.00797152, 0.009722453, 0.037489064, -0.017381987, -0.010079006, -0.0037883818, 0.002352617, -0.016707083, 0.009346798, 0.025022421, -0.016528806, -0.0095887445, -0.021202205, 0.021418683, -0.009964399, -0.0228831, -0.004915346, -0.0055679665, 0.005316469, 0.045817137, -0.021202205, 0.015752029, 0.009951665, -0.0054883785, -0.0071438057, -0.012390237, 0.006806353, -0.011479752, 0.02753103, -0.031351246, 0.011040427, 0.0046479306, 0.0020565502, -0.023468867, -0.020361757, 0.010677506, -0.013192482, -0.019572245, 0.013599972, -0.020794714, -0.036368467, -0.071463525, 0.018948276, 0.055673297, 0.016401464, 0.014580495, 0.011237805, 0.037387192, 0.03371978, 0.039602917, -0.02329059, 0.024907814, -0.0015272909, -0.03562989, -0.0037820148, -0.022832165, -0.015828433, -0.024385719, 0.003349057, -0.018668126, -0.01729285, -0.018196966, 0.019674117, 0.05091076, 0.018400712, -0.000497822, 0.005418341, 0.0037597301, 0.035986446, -0.006755417, 0.031096566, 0.012434807, 0.03560442, -0.027352754, 0.015802965, 0.033006676, 0.035578955, 0.008531818, 0.0024688153, 0.02712354, 0.008499983, 0.020221682, 0.03349057, -0.016567009, -0.04589354, 0.010683874, -0.03387259, 0.0060900627, 0.02476774, 0.00148113, -0.022080854, -0.02429658, -0.005195495, 0.003832951, 0.0045556086, 0.023647143, -0.044620134, 0.008920207, 0.006526204, 0.008200732, -0.013020573, 0.005427892, -0.027887585, -0.01002807, -0.044645604, -0.023736281, 0.020947523, -0.008321706, 0.029746756, 0.0023175983, -0.008277137, -0.066369906, -0.01660521, 0.004711601, 0.002510201, 0.028320542, 0.05198042, 0.0346621, -0.027607435, -0.011104098, -0.03130031, -0.0064338823, 0.017827678, 0.007653168, 0.009238558, -0.035833634, -0.009143053, -0.0069400608, 0.013676377, -0.015128059, -0.008894739, 0.0049567316, 0.032497313], "content_hash": "76bf20177d1be1a7a05f1c0028659224"}, {"content": "# Model Selector Guide\n\n## How to Choose a Model\n\nThe dashboard supports several transformer model families. You can select a model from the dropdown menu in the generator section at the top of the page.\n\n### Available Models\n\nCurrently, the dashboard offers:\n\n- **GPT-2 (124M)**: OpenAI's GPT-2 small model. 12 layers, 12 attention heads, 768-dimensional embeddings. This is the best model to start with -- it's small, fast, and well-studied.\n- **Qwen2.5-0.5B**: A LLaMA-like model from Alibaba's Qwen family. 24 layers, 14 attention heads, 896-dimensional embeddings. Slightly larger and uses different architectural features (RoPE, SiLU activation).\n\nYou can also enter a custom **HuggingFace model ID** in the dropdown (type it in). The dashboard supports GPT-2, LLaMA, OPT, GPT-NeoX, BLOOM, Falcon, and MPT model families.\n\n### What Happens When You Load a Model\n\n1. The model is downloaded from HuggingFace (this may take a moment the first time)\n2. The dashboard **auto-detects** the model's architecture family\n3. Internal hooks are automatically configured to capture attention patterns, MLP activations, and other data\n4. The layer and head dropdowns in the sidebar and ablation panel are populated based on the model's structure\n\n### Auto-Detection\n\nThe dashboard has a registry that maps model names to their architecture family. When it recognizes a model, it automatically configures:\n- Which internal modules to hook for attention capture\n- Which normalization parameters to track\n- The correct patterns for extracting layer outputs\n\nIf you enter an unknown model, the sidebar's configuration dropdowns may need manual adjustment.\n\n### Tips for Choosing", "source_file": "model_selector_guide.md", "chunk_index": 0, "embedding": [-0.014913806, -0.010687397, 0.035404727, -0.026242321, -0.006429867, -0.0060066036, -0.063340105, 0.011596168, -0.019358069, 0.058908287, -0.027014155, -0.040583476, -0.027586805, -0.004755487, 0.014540338, -0.008054451, 0.003992991, -0.012722796, -0.025520284, 0.049272824, 0.0041735, 0.008869855, 0.001178757, 0.0075502694, 0.015112988, -0.0032927392, -0.022196423, 0.030225975, -0.0028103436, -0.016743796, -0.0025442478, -0.03144597, -0.07867717, -0.012063003, -0.035653707, -0.014602582, 0.02325458, 0.03928879, -0.039712053, 0.081266545, 0.001583347, -0.009722606, -0.016918082, 0.07210414, 0.001895348, -0.0028212364, 0.006828232, -0.0007204812, -0.044442642, 0.02845823, 0.0017195069, 0.014702174, -0.051438935, 0.0016463696, 0.012834836, -0.0070087416, -0.033686776, -0.044168767, -0.012106574, 0.014639929, -0.031172095, 0.010289032, -0.0031340155, 0.057912372, 0.032591272, -0.04123082, -0.07568943, 0.014764418, -0.04872507, 0.003616411, -0.0055989013, -0.014067279, 0.023117643, 0.012417797, 0.03194393, -0.012448919, -0.02581906, 0.046932425, -0.026690483, -0.022955807, -0.007979757, -0.01681849, -0.020876838, -0.029304756, -0.017428488, -0.016059106, -0.03214311, -0.010264134, -0.04125572, -0.047704257, 0.0012394455, -0.018872561, -0.039487973, 0.028134558, 0.01762767, 0.012884632, 0.020876838, -0.041405104, -0.003020419, 0.06408703, 0.036898598, -0.009952911, 0.01739114, -0.009137507, 0.040832456, -0.041081432, 0.0368737, -0.008620877, -0.051737707, 0.012635653, -0.062443778, -0.024586616, 0.012100349, 0.016445022, -0.002099199, -0.015760332, -0.02557008, 0.020602962, 0.008900978, -0.03398555, -0.037521042, 0.009703932, 0.04269979, 0.011042192, 0.006296041, 0.0247609, 0.004223296, 0.0026485075, -0.014390951, 0.0061248685, 0.00053997186, 0.03296474, 0.028732106, -0.045911614, 0.0134697305, -0.024798248, -0.028532922, -0.027785989, -0.046484265, 0.0037253392, 0.034632895, -0.06647723, -0.013793402, 0.027337827, -0.07544045, -0.009037916, 0.0062120105, 0.0031713622, -0.0852502, 0.04319775, 0.00096323516, -0.027686397, -0.03030067, -0.02604314, -0.021051122, -0.016183594, -0.011857595, -0.00026901337, 0.02108847, -0.059953995, 0.015822576, 0.025258858, -0.01981868, -0.033935755, -0.054078106, 0.05502422, -0.0368737, -0.011614841, -0.041081432, -0.022370707, 0.026839871, -0.0041112555, 0.028059864, -0.011565046, 0.03602717, 0.0092370985, -0.040608376, -0.044044275, -0.024960084, -0.010413521, 0.0016992774, -0.01497605, -0.01168331, -0.0046776813, -0.0391643, 0.0073884334, -0.045089986, 0.014428297, 0.01658196, -0.021250306, -0.0066290493, 0.010730969, 0.034309223, 0.008340776, 0.009411383, 0.015137886, 0.0587589, 0.027462317, 0.033139024, -0.013569322, 0.0073697604, 0.020416228, 0.06343969, -0.08286001, 0.018735623, 0.02937945, -0.011876269, 0.025022328, -0.024350086, -0.023926822, 0.011944738, -0.042201836, 0.020229494, 0.009697708, -0.051438935, 0.0076685343, -0.017216856, -0.0033892184, 0.029404348, 0.05502422, 0.0055397693, 0.026989257, 0.018287463, 0.009859544, -0.0241758, 0.008832509, 0.023777436, -0.073647805, -0.011272497, -0.018299911, 0.025619876, 0.004101919, -0.013905443, 0.011116885, -0.0040054396, -0.03144597, 0.022196423, 0.0046558958, -0.02960353, -0.05597034, 0.013345242, 0.0028585831, 0.003762686, 0.003154245, 0.05238505, 0.0021225407, -0.007892615, -0.09764932, 0.040185113, 0.024462126, 0.033910856, 0.033363104, -0.0055864523, 0.014104625, 0.008241185, 0.016644206, -0.020690104, 0.017665017, -0.05572136, -0.049845472, -0.017565425, 0.05076669, 0.014963601, 0.011546372, -0.0156233935, -0.02131255, -0.05567157, -0.042102244, -0.057314824, -0.050293636, 0.02890639, -0.016681552, 0.012106574, 0.014191768, -0.027188439, -0.012834836, 0.019619497, 0.0049920166, -0.025794161, 0.0036381967, -0.009803524, -0.030375363, 0.062145006, 0.005256556, -0.006380071, 0.018648481, 0.013295446, -0.021399692, -0.02604314, 0.0059381346, 0.0046341103, -0.038267978, 0.016407676, -0.046384674, -0.013656464, 0.025669672, -0.022818869, 0.019495009, -0.009131283, 0.0069527216, 0.0059287976, -0.022632135, -0.022681931, -0.016743796, 0.053829126, 0.022520095, -0.016108902, 0.026192525, 0.014415849, 0.0050324756, -0.021486836, -0.030549647, 0.00033145247, 0.000758606, -0.031271685, 0.012834836, 0.0017552976, 0.0035137075, -0.0010986171, -0.02039133, -0.035081055, -0.02833374, -0.029927202, -0.015810128, -0.012959325, 0.020005414, -0.013258099, -0.02383968, -0.0074818004, -0.031993724, 0.041604288, -0.015262375, 0.011197803, 0.023279479, -0.017204406, -0.005922573, -0.012697898, 0.033462696, -0.02452437, 0.028582718, -0.043969583, -0.02234581, -0.039089605, -0.01670645, -0.06408703, -0.012797489, -0.000419762, 0.004114368, 0.0155487005, -0.025184164, 0.0032149334, -0.008714244, -0.011565046, -0.014129523, 0.0035728398, 0.039089605, -0.0112849455, -0.020105004, -0.023105195, -0.018835215, -0.00011777845, 0.040185113, 0.02096398, 0.0012721239, -0.019756434, -0.009181078, 0.0038809506, 0.030798627, -0.0072079245, -0.026690483, -0.0077432278, 0.00030636013, 0.008347, -0.01634543, -0.04364591, -0.040633272, 0.022034587, -0.012311981, -0.016034208, 0.037819818, 0.062443778, 0.008658224, 0.019656844, -0.059804607, 0.045587942, 0.007861492, 0.020827042, -0.018723175, -0.030051691, -9.861878e-05, 0.008284756, -0.021748262, -0.020814594, 0.0010752754, 0.0011554153, 0.035056155, 0.013656464, -0.024200698, 0.05208628, 0.03157046, 0.031196991, -0.021449488, -0.03488187, -0.0056798193, -0.019171337, -0.0032180457, -9.409632e-05, 0.026317015, 0.01145923, -0.023802334, 0.052733622, -0.038990017, 0.021735813, 0.009398934, -0.026914563, -0.0026096047, 0.014652378, 0.031221889, 0.013046468, -0.0114530055, -0.032317396, -0.07070986, -0.04309816, -0.022793971, -0.06777192, -0.00014043937, 0.04675814, -0.03754594, 0.005135179, -0.030450057, 0.0036257477, 0.010201889, 0.037819818, -0.039961033, 0.022644585, -0.055472385, 0.044990394, 0.008434143, 0.034483507, 0.008595979, 0.028034966, 0.012884632, 0.04043409, -0.013345242, 0.012311981, -0.022968257, -0.051787503, -0.02248275, -0.061597254, -0.036574926, -0.02776109, 0.046608754, 0.0050418125, -0.009218425, -0.00091966393, -0.031246787, 0.07066007, -0.006977619, 0.07110823, -0.022818869, 0.032541476, 0.02050337, -0.0027574357, 0.006660172, -0.035852887, 0.00127368, 0.030972911, -0.018312361, 0.019196233, -0.037072882, 0.006404969, -0.035852887, 0.010724744, -0.014478093, -0.018411951, -0.018474197, 0.036126763, 0.006921599, -0.00046916865, -0.02741252, -0.0021443265, 0.0017786394, 0.00956077, 0.015710536, 0.025744366, 0.10327624, 0.009473627, 0.024474576, 0.00972883, 9.297787e-05, -0.03667452, -0.012747694, 0.039886337, -0.01238045, 0.049596496, -0.03478228, -0.015922168, -0.04021001, -0.05358015, 0.00039661478, -0.018561339, -0.015735434, 0.010998621, 0.06822008, -0.014851561, -0.013220753, 0.0134572815, 0.017129714, -0.02039133, 0.015735434, -0.0029021543, 0.008042002, -0.018636033, 0.040160213, -0.0010908366, 0.018287463, 0.032765556, 0.035429623, 0.024810696, 0.022159077, -0.023652947, -0.011303619, -0.039687153, -0.006840681, 0.009759952, -0.01659441, -0.024449678, -0.073199645, -0.025059674, -0.016270738, 0.029902304, -0.023466213, -0.022495197, -0.032516576, -0.059256855, 0.015685638, -0.0039494196, -0.041305516, -0.046932425, 0.019980516, -0.02291846, 0.019158887, 0.023628049, 0.004715028, 0.0016308085, 0.02213418, 0.0019996078, -0.013793402, 0.0040583475, 0.035379827, 0.026441505, -0.018125627, 0.037521042, -0.007245271, 0.012604531, 0.002885037, -0.004425591, 0.006510785, 0.018050933, -0.039413277, 0.00044349275, 0.008421694, -0.003059322, -0.02223377, 0.010967498, 0.025495388, -0.016445022, -0.018723175, 0.0047368137, -0.034010448, -0.017901545, -0.030400261, 0.03169495, 0.03627615, -0.0026158292, 0.029279858, 0.023665396, 0.03951287, 0.016532164, -0.0064111934, -0.043845095, -0.027910477, -0.020428676, 0.011932289, -0.0036101865, 0.021287652, -0.018063381, -0.008042002, -0.013345242, 0.009703932, 0.037745122, -0.025769264, -0.011004845, 0.0089321, -0.03699819, -0.03440881, 0.014801765, -0.0067784367, -0.041604288, -0.018138075, 0.016046656, 0.005321913, -0.0038000327, 0.008172715, 0.005822982, -0.00690915, -0.01899705, 0.01810073, -0.011247599, -0.025159266, -0.031022707, -0.014092176, -0.019731537, -0.008129144, -0.046085898, 0.028781902, -0.040832456, 0.009056589, 0.049820576, -0.0011165125, -0.002194122, -0.033238616, 0.021325, -0.0037751347, -0.0113658635, 0.018523991, 0.006068848, -0.0073884334, -0.034956567, 0.021262754, -0.03791941, -0.0034607996, 0.062244598, 0.00771833, -0.019544803, 0.012461368, 0.007444454, -0.056717277, 0.03396065, -0.0071270065, 0.0002289434, -0.0020929745, -0.0077681257, -0.010525561, 0.013644015, -0.0030453168, -0.005863441, 0.02950394, -0.028034966, -0.0080669, -0.029628428, -0.031993724, 0.047380585, 0.015237478, 0.015685638, 0.0075627184, 0.033487592, -0.008340776, -0.020690104, 0.044442642, -0.07220373, 0.016108902, -0.016756246, -0.041056536, 0.0025115695, 0.039961033, 0.025794161, 0.01589727, 0.014851561, -0.024038862, -0.026939461, 0.009162405, 0.0051911995, -0.014204217, -0.029528836, -0.03719737, 0.017577874, -0.03547942, -0.0077121053, 0.028184352, -0.031620257, 0.016108902, -0.056767073, 0.0155362515, 0.03478228, -0.008620877, -0.005144516, -0.010046278, 0.019507457, 0.008284756, -0.012921978, -0.041728776, 0.0068469057, -0.031097401, -0.02178561, 0.013295446, -0.016606858, 0.06458499, -0.018250115, -0.0702119, -0.027910477, 0.016644206, -0.0041112555, -0.0025255745, -0.018187871, 0.0015903495, -0.04307326, 0.039438177, 0.018312361, -0.017677465, -0.023814783, 0.018586237, 0.014677276, -0.008421694, -0.0029970773, -0.0018113178, -0.007985982, 0.02751211, -0.011558821, -0.0035168198, -0.04757977, 0.026590891, -0.025321102, 0.009442505, 0.021125816, -0.0345831, -0.0023481774, 0.0013771617, -0.039438177, 0.01832481, 0.011154232, 0.013531975, -0.022059485, 0.0043259994, 0.0077992477, 0.0035137075, 0.015959514, 0.005882114, -0.047056913, -0.021673568, -0.01833726, 0.023291929, -0.021499284, 0.025520284, 0.01221239, 0.005468188, -0.013581771, 0.022071933, 0.003955644, -0.031097401, 0.011216477, 0.009261996, 0.008315878, -0.005944359, -0.0060066036, -0.0054713, 0.04630998, -0.030325567, -0.004195286, 0.03983654, 0.036599822, 0.013780953, -0.010295256, 0.031122299, -0.023814783, -0.004207735, 0.00882006, 0.020889286, 0.00024975644, -0.023067847, -0.0031355715, 0.005935022, 0.011926064, 0.008689346, 0.027611703, -0.04135531, 0.022569891, 0.015810128, -0.012847285, 0.0042855404, -0.01151525, 0.020988878, 0.01186382, -0.0138183, 0.015337069, -0.012984223, -0.0114094345, -0.03418473, -0.006442316, -0.00956077, -0.012679224, 0.024897838, 0.0154740065, -0.0057265027, 0.013046468, 0.010569133, 0.007544045, -0.11034722, -0.05930665, 0.010220563, -0.022544993, 0.008751591, -0.00794241, 0.025868854, 0.018461747, 0.033263512, -0.045065086, 0.017366242, -0.01991827, 0.036002275, 0.01833726, 0.025520284, -0.04790344, 0.01694298, -0.011782902, 0.048924252, -0.01659441, -0.031520665, -0.0006699075, 0.014988499, 0.018611135, -0.04643447, -0.028657412, -0.012847285, 0.002259479, 0.042799383, 0.0036537577, -0.045687534, 0.019382967, -0.0022283567, 0.030674137, 0.00841547, 0.0414549, -0.0010892805, -0.021486836, 0.0044847233, -0.030001896, -0.030250873, -0.021100918, 0.006616601, -0.04653406, 0.02545804, 0.0072079245, 0.013594219, 0.011745555, -0.02442478, 0.020988878, 0.021163164, 0.00028457452, 0.007898839, 0.04409407, 0.00033553728, -0.009647912, -0.001651038, -0.011658413, -0.010226787, -0.0006745758, 0.0038218182, 0.035653707, 0.013594219, 0.021038674, 0.007363536, 0.005306352, 0.04006062, 0.010618928, -0.034483507, -0.013158508, -0.040956944, 0.024262944, 0.00066251593, 0.0025286868, -0.008888529, -0.005956808, 0.022109281, 0.03821818, 0.0012962437, -0.029479042, 0.026665585, -0.005175638, -0.05482504, 0.009579443, 0.012125247, 0.0357035, 0.012287083, -0.027960273, 0.021524182, 0.018250115, 0.010656275, 0.009741279, 0.009890666, -0.0070647616, -0.018561339, -0.00766231, -0.017665017, 0.045861818, 0.02441233, -0.02581906, 0.050393224, 0.022395605, -0.026590891, -0.025420694, -0.018760521, -0.017702363, 0.017366242, -0.022034587, -0.024935186, 0.008197613, 0.0019435876, 0.03525534, -0.021611324, 0.032566372, 0.011533923, 0.034707587, 0.06294174, 0.035230443, 0.04068307, 0.008315878, 0.04364591, -0.026117833, -0.009915564, 0.027611703, -0.01646992, 0.0022890451, 0.01945766, 0.02545804, 0.012722796, 0.019158887, -0.0035852888, 0.016880734, 0.002559809, -0.015112988, -0.009585668, 0.00029235508, 0.016843388, -0.021051122, -0.022769073, -0.0074382294, -0.011658413, 0.00017311779, -0.0052036485, -0.013345242, -0.0030422048, 0.00047733824, 0.00049367745, -0.05024384, 0.0070336396, -0.0046901302, 0.020814594, 0.0033798816, -0.028931288, 0.048401397, 0.028632514, 0.0018377717, -0.003912073, -0.038392466, -0.014951152, 0.015125437, 0.046334878, 0.006660172, 0.039886337, -0.026914563, 0.034259427, -0.011583719, 0.003641309, 0.052235667, -0.014029932, 0.012492491, -0.0046216613, 0.014901357, 0.042998567, 0.021997241, 0.03510595, 0.045812022, -0.0017226192, 0.018922357, 0.0046590082, 0.025769264, -0.01785175, 0.07110823, 0.027935375, 0.028034966, 0.019258479, 0.005184975, -0.030126385, -0.0057047172, -0.011677086, 0.010276583, 0.003442126, 0.0041392655, -0.0067908857, 0.011266272, 0.0035915133, -0.016395226, 0.036699414, 0.00050184707, -0.031421073, -0.0016448135, -0.039214097, 0.03296474, 0.0068780277, -0.011035967, -0.01462748, 0.009205976, 0.009486076, 0.004820844, -0.014141972, 0.0074382294, -0.028308842, 0.014552787, -0.0134199355, 0.013569322, -0.0008340776, 0.009934237, 0.0357284, 0.0041641635, -0.016781144, -0.0066539473, -0.06961436, -0.047256097, -0.018573787, -0.019519905, 0.0023077184, -0.004811507, 0.0064111934, 0.023677845, 0.007270169, 0.010612704, 0.026790075, -0.017951341, 0.024088658, 0.00077922456, -0.010892805, 0.018362155, 0.029852508, 0.0073199645, 0.0019575926, 0.01532462, 0.029553734, 0.022420503, -0.0044567133, -0.023640499, 0.020329086, -0.011148008, 0.02050337, -0.010245461, 0.038068794, -0.04561284, 0.0006282814, 0.0070772106, 0.031620257, -0.038541853, 0.0028305731, 0.020764798, 0.021748262, 0.033014532, 0.025084572, -0.020080106, 0.0030639903, 0.028034966, 0.0076125143, -0.03194393, 0.01588482, 0.009791075, -0.0046278858, 0.00070881034, 0.020715002, 0.001800425, -0.032815352, 0.03510595, -0.027387623, 0.00203073, -0.0052223215, 0.0033238614, 0.029279858, 0.0134821795, 0.007936186, -0.009679034, -0.008135369, -0.021038674, -0.009666586, 0.0032149334, 0.004478499, 0.007556494, -0.026317015, 0.015810128, 0.021735813, -0.019308275, -0.020715002, 0.020067658, 0.0046807937, -0.020403778, -0.022669483, 0.028707208, -0.009181078, 0.0026220537, 0.001854889, -0.03976185, 0.029130472, -0.0076311873, 0.0040552355, -0.053629946, 0.006404969, -0.009710157, 0.0113285165, -0.05502422, 0.003709778, 0.033562288, -0.026117833, 0.017615221, 0.0038902874, 0.03236719, 0.02166112, 0.014577685, -0.009604341, 0.0026594005, 0.00078622706, -0.003101337, 0.021984791, 0.017117264, -0.001665043, -0.0074133314, -0.01762767, -0.018374605, -0.0039774296, 0.0144034, -0.009834646, 0.020490922, -0.0024150903, -0.0019544803, 0.027785989, 0.040409192, -0.04596141, -0.041504696, -0.049098536, -0.04001083, 0.02061541, 0.008776488, 0.01203188, 0.022445401, -0.030599443, 0.0028259046, 0.033811264, 0.0021474385, 0.015187682, -0.017079918, 0.021686018, 0.016843388, 0.03582799, -0.022569891, -0.024101108, -0.021113368, 0.004743038, 0.015847474, -0.015461558, 0.010419746, -0.007849043, -0.023877027, 0.02466131, 0.023802334, -0.033014532, 0.021175612, 0.026068037, -0.014104625, -0.029329654, -0.047480177, 0.028881492, 0.005564667, 0.024200698, 0.012884632, -0.01416687, -0.00026317794, 0.030151283, 0.022532543, 0.009000569, -0.013644015, 0.0028227924, 0.03042516, -0.0062867044, -0.06836946, 0.010681173, 0.016731348, -0.006784661, 0.007998431, -0.0071270065, 0.013830749, -0.017963791, 0.006498336, -0.000263956, -0.028034966, 0.009504749, -0.0063022655, 0.006442316, -0.0008426362, -0.0043135504, 0.009118834, -0.0020385105, 0.0026049365, -0.004157939, -0.0051227305, -0.02706395, -0.011614841, 0.025283756, -0.03789451, -0.0034483506, 0.027686397, 0.021698466, 0.021001328, -0.03398555, 0.033263512, -0.016096452, 0.017814403, 0.028034966, -0.026317015, -0.01129117, 0.03388596, 0.022432953, 0.05054261, -0.022333361, 0.00628048, 0.0036475332, -0.02903088, 0.0495467, 0.023752538, -0.024213148, -0.01833726, 0.0299521, 0.012598306, 0.006121756, -0.0024742228, -0.012548511, 0.017503181, -0.02407621, -0.008857406, 0.0288068, -0.010176991, -0.027586805, 0.023055399, 0.0029488378, 0.027910477, 0.021959893, 0.052285463, -0.00080373336, 0.0113783125, -0.038417365, -0.0053654844, 0.032665964, 0.0109426, 0.016743796, 0.0027636602, -0.030699035, 0.019084193, -0.011471679, 0.019482559, -0.009691483, -0.02639171, 0.026964359, -0.005147628, -0.021972343, 0.00051818625, 0.0041112555, 0.007120782, 0.0073448624, 0.015959514, -0.006121756, 0.05218587, -0.004024113, -0.036101867, 0.029429246, -0.011540148, 0.006000379, -0.0059101246, 0.019955618, -0.028184352, -0.004995129, -0.01117913, -0.03983654, -0.009934237, 0.008788937, 0.027213337, 0.003073327, -0.029130472, -0.0028694759, 0.003034424, -0.006199562, -0.03216801, -0.01302157, 0.008764039, 0.019395417, 0.027661499, -0.004279316, 0.00019286099, 0.012498715, 0.008297205, 0.015722984, -0.0050387, -0.017503181, 0.0027916702, 0.0071705775, -0.03674921, 0.024922736, -0.022159077, -0.0027589917, -0.03513085, 0.0037533492, -0.015561149, -0.012704122, -0.022084383, -0.008222511, 0.008602203, -0.016694, -0.0077619012, -0.01302157, -0.030126385, -0.0133950375, 0.01123515, 0.021710915, 0.035429623, 0.013245651, -0.0005481415, -0.01808828, -0.005377933, 0.018723175, -0.00904414, -0.012834836, -0.02037888, -0.029354552, 0.030375363, 0.007419556, 0.026541095, -0.009679034, -0.023578253, -0.007083435, 0.02107602, -0.033064328, 0.0028414659, 0.0073075155, -0.003535493, -0.010556684, -0.045314066, 0.00023905815, -0.008284756, 0.008110471, 0.027661499, -0.013917892, 0.024026414, 0.025346, -0.0030281995, 0.004851966, 0.010289032, -0.018561339, -0.018623583, -0.056269117, 0.017876647, 0.000622835, 0.029105574, 0.007593841, 0.019943168, -0.0030904443, -0.02751211, -0.021835405, 0.005013802, 0.04479121, -0.0012231063, 0.014727072, -0.016606858, 0.013892994, -0.00990934, 0.017191958, -0.016868286, 0.037794918, 0.009274445, -0.0031200105, 0.03861655, 0.008421694, -0.024474576, 0.038765933, 0.019631946, 0.008919651, -0.031271685, 0.004266867, 0.0154117625, -0.023827232, -0.012006982, -0.043372035, 0.01168331, -0.0038560526, 0.0052721174, -0.012984223, 0.02557008, 0.0035510543, 0.035379827, -0.0035852888, 0.04449244, 0.03214311, -0.002044735, -0.04790344, 0.011894942, -0.011801575, 0.012716571, -0.019831128, -0.006429867, -0.041405104, -0.01106709, -0.0093429135, 0.0039369706, -0.00972883, 0.009871993, -0.011982085, 0.0041735, 0.015299722, 0.02061541, 0.016519716, 0.053032394, 0.0010441531, -0.016046656, 0.0018611135, -0.009604341, -0.008166491, -0.026814973, -0.017378692, 0.005935022, 0.0014806433, 0.037819818, -0.012996672, -0.007189251, 0.0043540094, 0.010668724, -0.003737788, 0.018785419, -0.011621066, -0.0133950375, 0.0072079245, 0.0379941, 0.030923115, -0.012467593, -0.04075776, -0.007245271, -0.0112973945, 0.026790075, 0.009517198, -0.025744366, -0.0019513681, 0.023528457, -0.0092122005, -0.023640499, -0.051438935, 0.04225163, -0.0056673707, -0.02441233, 0.003358096, -0.0047897217, 0.044890802, 0.019071745, -0.0066290493, 0.03176964, -0.000345263, -0.018063381, 0.010276583, -0.018947255, -0.037072882, -0.018362155, 0.017229304, -0.014702174, 0.009554545, 0.020229494, -0.0114405565, 0.031819437, -0.008764039, 0.015013397, 0.020914184, 0.006134205, 0.059705015, 0.00829098, 2.484921e-05, 0.011907391, 0.05163812, -0.027537009, 0.0001231276, 0.026341913, 0.02349111, 0.027437419, -0.018299911, -0.0010877243, -0.017652567, 0.0089321, 0.047380585, -0.012554735, -0.0058727777, -0.0015312171, -0.04805283, -0.010618928, -0.010033829, 0.024325188, 0.0035852888, -0.01751563, 0.009019242, -0.0076560853, 0.022706829, 0.011677086, -0.011614841, 0.014291359, 0.021212958, 0.0052752295, -0.010071176, -0.005035588, -0.017341344, -0.020652758, -0.034682687, -0.017440936, 0.016295636, -0.003208709, 0.00047033574, 0.028433332, -0.030748831, -0.054675654, -0.014615031, -0.054874837, -0.0036755435, 0.027686397, 0.03191903, -0.039712053, -0.02477335, -0.014303808, 0.027039053, -0.023005603, 0.01922113, -0.0031028932, 0.014117074, -0.008347, -0.011670861, -0.009131283, 0.020515818, -0.0033207492, -0.0026360587, 0.0019575926, 0.00066640624], "content_hash": "968306d458b1086d2717b6ad6a72370d"}, {"content": "family. When it recognizes a model, it automatically configures: - Which internal modules to hook for attention capture - Which normalization parameters to track - The correct patterns for extracting layer outputs If you enter an unknown model, the sidebar's configuration dropdowns may need manual adjustment. ### Tips for Choosing\n\n- **Start with GPT-2**: It's small, fast, and the most widely studied. Most educational resources reference GPT-2.\n- **Try Qwen2.5-0.5B for comparison**: It uses a different architecture (LLaMA-style). Comparing results between GPT-2 and Qwen can highlight how architectural differences affect attention patterns.\n- **Larger models are slower**: Models with more parameters take longer to load and analyze. Stick to small models for interactive exploration.\n- **Memory matters**: Larger models require more RAM. If the dashboard becomes unresponsive, try a smaller model.\n\n### Generation Settings\n\nAfter selecting a model and entering a prompt, you can configure:\n- **Number of Generation Choices (Beams)**: 1-5 beams. More beams explore more paths but take longer.\n- **Number of New Tokens**: 1-20 tokens to generate. Shorter is faster.\n\nClick **Analyze** to run the model and see results in the pipeline and generation sections.", "source_file": "model_selector_guide.md", "chunk_index": 1, "embedding": [-0.0010694269, 0.024276784, 0.06263156, -0.028706133, -0.045563053, -0.018465027, -0.01345027, 0.03244428, 0.004990071, 0.076229945, -0.009387681, -0.021681242, -0.022372445, -0.024431953, 0.010071833, -0.029256275, -0.0062843156, -0.011567091, -0.00021049107, 0.04784826, 0.013831137, 0.0072294315, -0.015389873, 0.008104016, 0.042798236, -0.0024703497, -0.00808991, 0.022513509, 0.017124936, -0.00758914, 0.008985654, -0.028141886, -0.06928969, -0.032669976, -0.054703876, -0.013478482, -0.015714316, 0.022739207, -0.015601466, 0.04119013, -0.0015895934, -0.016306777, -0.031118294, 0.04333427, 0.021864623, 0.0057341736, 0.008273291, -0.0054978947, -0.04040018, 0.041161913, -0.019001063, 0.054224264, -0.041274764, 0.020665595, -0.004549252, -0.037635364, -0.036619715, -0.04471668, -0.0027701065, 0.01663122, 0.022866163, -0.0044575618, 0.0118985865, 0.056255557, 0.044773106, -0.045365565, -0.058004726, 0.020524533, -0.053180404, 0.017689185, 0.0021335639, -0.001670704, -0.017237786, -0.03128757, 0.002428031, 0.017914884, 0.02101825, 0.027295513, -0.006746294, -0.02347273, 0.0138452435, -0.032303218, -0.0024174515, -0.02533475, 0.0032514809, -0.011207382, 0.0009310097, -0.02510905, -0.017237786, -0.06985394, -0.007257644, 0.016574794, -0.04426528, 0.039751295, 0.037437875, 0.002350447, 0.02585668, -0.06471928, 0.0016901001, 0.030328346, 0.023289349, -0.014614032, -0.033459924, 0.003960318, 0.039694868, -0.03597083, -0.00026779756, -0.001987212, -0.055127062, 0.0053815185, -0.07639922, -0.00049107237, -0.006644024, 0.0063901125, 0.01756223, -0.022005685, -0.029397337, 0.008506044, 0.04130298, -0.021243948, -0.03492697, 0.027775124, 0.0019925018, 0.008336769, 0.023077756, 0.019339612, 0.021201631, -0.005864656, -0.0008878095, -0.00503239, -0.009930771, 0.03523731, 0.0411337, -0.019339612, -0.02674537, -0.017548123, -0.008929229, 0.01054439, -0.047537923, -0.0011549458, 0.009803815, -0.031456843, 0.001351551, 0.048412506, -0.09575294, 0.016772281, -0.029989798, 0.045929812, -0.06765337, 0.054732088, 0.0064289044, -0.004535146, -0.023599686, -0.033008527, 0.007398706, -0.032303218, -0.017844353, 0.015855378, -0.004856062, -0.051459447, 0.024601227, 0.035519432, -0.03035656, -0.0074269185, -0.044914167, 0.080405384, -0.046606913, 0.016786387, -0.042487897, -0.020693807, 0.016828706, 0.01547451, 0.013852296, -0.02556045, 0.02242887, -0.018253433, -0.094173044, -0.034475572, -0.046578698, -0.060036022, -0.022499401, -0.020877188, -0.038002126, -0.0052016643, -0.030582258, -0.016645325, -0.018408602, -0.017435273, 0.023712536, -0.015643785, 0.016419627, 0.0044258228, 0.07713275, -0.0077090426, 0.005945767, -0.034616634, 0.07544, 0.02072202, -0.00031165904, -0.024079297, -0.0006581428, -0.013915774, 0.043842096, -0.07848694, 0.03233143, 0.018648406, 0.015333449, 0.016800493, -0.04516808, -0.005423837, 0.02692875, -0.06364721, 0.043616395, 0.0017077328, -0.051515874, 0.01415558, -0.021441435, 0.005155819, 0.02919985, 0.047368646, 0.0049406993, 0.0021564865, 0.03876386, 0.022146747, -0.006139727, 0.0340806, 0.033657413, -0.048412506, 0.0120960735, -0.026759477, 0.010579656, 0.016052864, -0.015107749, -0.013422057, 0.0084073, -0.0026149384, 0.0001881195, -0.0050676554, -0.00513466, -0.06426788, 0.0036887734, 0.007786627, -0.0028600337, -0.003935632, 0.03233143, 0.025574556, -0.027394256, -0.079389736, 0.037522513, 0.014628137, 0.00010485982, -0.014585819, -0.014070943, 0.011002842, 0.020933613, -0.007800733, -0.02797261, 0.012794331, -0.038509946, -0.019819222, -0.0013321551, 0.037296813, 0.013647757, -0.029820524, -0.023613792, -0.021794092, -0.051797997, -0.051515874, -0.042911086, -0.042346835, 0.040089842, -0.050133463, 0.0046832613, -0.004951279, -0.054337114, -0.0043023936, 0.01774561, -0.0074903965, -0.021624817, 0.0009812631, -0.050951622, -0.008957442, 0.0767942, 0.024544802, -0.022273703, 0.012258295, 0.02392413, -0.003041651, -0.01395104, 0.011962065, 0.02306365, -0.039638445, 0.0071835862, -0.015164173, -0.0023098916, -0.011806896, -0.015883591, 0.016659431, -0.0023592634, -0.009754443, 0.005977506, -0.010741877, 0.004983018, -0.03619653, 0.022866163, 0.034955185, -0.0005787953, 0.017618654, 0.032782827, 0.020044921, -0.009190194, -0.008216866, 0.00054837886, -0.014487076, -0.006612285, -0.01565789, 0.014853837, 0.00082433154, 0.026350396, -0.022344233, -0.024558907, -0.043842096, -0.014261376, -0.0051064473, -0.0017553413, 0.009084398, -0.006996679, -0.009260725, 0.01722368, -0.031061871, 0.02492567, 0.018408602, -0.004210703, 0.039694868, 0.007204746, -0.051092684, -0.007786627, 0.037437875, -2.6614447e-05, 0.044293493, -0.08096963, -0.00954285, -0.016433733, 0.012328826, -0.0497667, 0.023642005, 0.0021564865, 0.035716917, 0.06364721, -0.026477352, 0.013732393, -0.0012051992, -0.041867226, -0.022104427, 0.03549122, 0.029115213, 0.0038827339, -0.02797261, -0.01129202, -0.0041119596, -0.004746739, 0.039074197, 0.05478851, 0.02242887, -0.009944877, -0.009063238, -0.036845416, 0.016941557, -0.0011320232, -0.004281234, -0.021229843, 0.006485329, -0.0054273633, -0.0036182425, -0.06528353, -0.031033657, 0.004312973, 0.03772, -0.019579416, 0.053208616, 0.047001887, -0.021779986, 0.022964906, -0.037635364, 0.029143427, 0.026054166, 0.00019175626, 0.01056555, -0.06071312, -0.0045915707, 0.011334338, -0.003239138, -0.017816141, -0.008978602, 0.0064430106, 0.039864145, 0.00080140895, 0.0060021915, 0.0470301, 0.041613314, 0.027732804, -0.016137501, -0.021554286, 0.008456672, 0.007744308, 0.016786387, 0.020834869, 0.042290412, 0.05684802, -0.027182663, 0.050613075, -0.03506803, 0.027549425, -0.0008745849, -0.016208032, 0.0023557367, 0.018902319, 0.023867704, 0.018140584, 0.02056685, -0.01622214, -0.0767942, -0.04051303, -0.023373988, -0.040795155, -0.0013356815, 0.026759477, -0.023867704, 0.017209575, -0.01670175, -0.018648406, 0.036845416, 0.060205296, -0.021258056, 0.013774713, -0.043983158, 0.033206012, 0.013027083, 0.01804184, -0.005000651, 0.017181361, -0.0025902526, 0.01077009, -0.0020348206, -0.009380628, -0.0012184237, -0.094624445, -0.03715575, -0.018211115, -0.055380974, -0.020369364, 0.026350396, -0.0068379845, 0.025193688, 0.0056918547, -0.026209334, 0.07244948, -0.01074893, 0.07967186, 0.0072999625, 0.035406582, 0.00564601, -0.009683912, 0.021483755, -0.019805117, 0.027140344, -0.016673539, 0.005416784, 0.026787689, -0.0054449965, 0.015601466, -0.03142863, -0.008513097, -0.046635125, -0.013464375, -0.03622474, 0.0087246895, 0.014726881, 0.0074269185, -0.033459924, 0.0010103572, 0.036986478, 0.046889037, 0.06167234, -0.015432192, 0.102128945, -0.01886, 0.011143904, 0.01199733, 0.016574794, -0.04003342, -0.026604308, 0.035519432, -0.01733653, 0.034362722, -0.03255713, 0.007673777, -0.0060480367, -0.022640465, 0.0010165286, -0.0005219297, -0.0012157788, -0.004083747, 0.041669738, -0.033713836, -0.0131963575, 0.011306126, 0.009874346, 0.0032426645, 0.014487076, -0.04025912, -0.001699798, -0.011743418, 0.049399942, -0.023049545, 0.03281104, 0.032839254, 0.034024175, 0.017139044, 0.024488376, 0.009063238, -0.011228542, -0.032613553, 0.009874346, 0.0010817698, -0.060205296, -0.03642223, -0.080236115, -0.03921526, -0.018775363, 0.042487897, -0.037861064, -0.022852058, -0.022160852, -0.08096963, 0.006746294, -0.0043306057, -0.0137465, -0.036252953, 0.0043905573, -0.008082857, 0.033234227, 0.013415004, 0.004446982, -0.015883591, 0.033685625, 0.02053864, -0.004260075, -0.001702443, 0.02919985, 0.024460165, -0.05007704, 0.022344233, -0.031061871, 0.010438594, 0.020468108, 0.0051064473, 0.00082256825, -0.011644674, -0.035406582, 0.0023539735, 0.0039850036, 0.013817031, -0.040428393, -0.019113911, 0.035886195, 0.0014159107, -0.03035656, 0.013104667, -0.023289349, -0.0125474725, -0.027196769, 0.016532475, -0.0047502657, 0.014240217, -0.0025779095, -0.0147550935, 0.033911325, -0.002031294, 0.0052545625, -0.0068908827, -0.021568391, -0.0026025954, -0.009387681, -0.026590202, 0.0040908004, 0.0023751329, 0.01874715, -0.016969768, 0.029989798, 0.043362483, -0.02038347, 0.005258089, 0.007596193, -0.048497144, -0.0168146, 0.0009186668, -0.021991579, -0.041613314, -0.010241107, 0.013076455, 0.0063619, -0.020482214, 0.010234054, -0.0048454823, 0.0018919952, -0.019664055, 0.01026932, -0.005367412, -0.0050747083, -0.024417846, -0.0118985865, -0.022513509, -0.014233164, -0.024488376, 0.011306126, -0.011468347, -0.005811758, 0.0470301, 0.007850105, 0.01990386, -0.0252219, 0.027493, -0.0037839904, 0.012004383, 0.017646866, -0.023811279, -0.00687325, -0.019480674, 0.028395796, -0.028409904, -0.009881399, 0.032303218, 0.0030063856, -0.02842401, 0.008745849, 0.019720478, -0.005007704, 0.0084637245, -0.022569932, 0.024742289, 0.010078886, -0.018775363, -0.025066731, 0.0054626293, 0.006527648, -0.005212244, 0.010191735, -0.016729962, -0.017139044, -0.047594346, -0.029707674, 0.05081056, 0.010741877, 0.018239327, -0.018055946, 0.0024491905, -0.016969768, -0.012067861, 0.026886433, -0.055437397, 0.03326244, -0.034701273, -0.04601445, 0.008484884, 0.047453284, -0.00054926047, 0.025165476, -0.011150957, -0.0114471875, -0.04076694, 0.01997439, 0.009183141, 0.00287414, -0.017082619, -0.026181122, -0.0032162154, -0.053998563, 0.006432431, 0.036647927, -0.018197007, 0.0014511761, -0.0586254, 0.010897046, 0.034503784, -0.0017553413, 0.010262267, -0.0041366457, 0.028254734, -0.00696494, 0.02108878, -0.02849454, 0.00022360544, -0.001466164, -0.03128757, 0.0022023318, -0.0051734517, 0.058061153, -0.0069226217, -0.052136544, -0.04200829, 0.02254172, -0.01197617, -0.01229356, -0.016603006, -0.016645325, -0.036140107, 0.036958266, 0.03213394, -0.0021564865, -0.028861301, 0.012773171, -0.0065452806, 0.00615736, -0.015855378, -0.005480262, -0.009147876, 0.024417846, -0.00068326946, -0.015389873, -0.04110549, 0.030554047, -0.028184203, 0.009190194, 0.00095657725, -0.03735324, 0.009507584, -0.0020789024, -0.027210874, 0.04437813, 0.030695109, 0.012871915, 0.005117027, -0.0076314583, -0.014825624, 0.023811279, -0.013104667, 0.0073422813, -0.009006814, -0.014395385, 0.006541754, 0.020580957, -0.010191735, -0.0005937832, 0.019551205, -0.00066211016, -0.0050958674, 0.048440717, 0.015756635, -0.058174003, -0.010763037, 0.01324573, 0.00093629956, 0.0036394016, 0.010819461, -0.01904338, 0.039723083, -0.0044328757, 0.002542644, 0.02942555, 0.03526552, -0.0042636013, -0.00225523, 0.022527615, -0.016758176, 0.004503407, -0.0009715651, 0.018422708, 0.0035530012, 0.009288938, -0.0003780023, -0.019522991, 0.020284727, 0.014642244, 0.04322142, -0.031090083, 0.03828425, 0.011263807, -0.0050888145, 0.02488335, -0.002204095, 0.012244188, -0.00933831, -0.016461944, 0.004126066, -0.016913343, -0.0032250318, -0.02165303, -0.01054439, -0.009599275, -0.007155374, 0.007028418, -0.0007590903, -0.009147876, 0.012413464, -0.017294211, -0.009585168, -0.12876147, -0.025193688, 0.014014518, -0.029623037, -0.0038157292, -0.036027256, 0.042713597, 0.006612285, -0.023232924, -0.01629267, 0.000115163966, -0.020707913, 0.0043799775, -0.0061926255, 0.027253194, -0.07888191, 0.000983908, -0.0011699336, 0.045845177, -0.015418085, -0.023783067, -0.015545041, 0.0018108845, 0.023910023, -0.04624015, -0.009719177, -0.00831561, 0.011799843, 0.0061608865, 0.0009847897, -0.050133463, 0.051403023, -0.0043799775, 0.031259358, -0.0044822474, 0.019438354, -0.020595064, -0.00030769166, 0.019128019, -0.019029275, -0.017971309, -0.014684563, 0.008625946, -0.026731264, 0.008421406, 0.018718937, 0.015135962, 0.0028582704, -0.032669976, 0.026999282, 0.011242648, -0.006957887, 0.011150957, 0.014938475, -0.0021617764, 0.017463485, 0.004006163, -0.015065431, 0.0058928686, 0.019297292, -0.0013991595, 0.03072332, 0.017914884, -0.01706851, -0.004549252, -0.00078553945, 0.035463005, -0.00096098543, -0.028974151, -0.015883591, -0.046691548, 0.019212656, 0.0068591437, -0.0020718493, -0.016800493, 0.0127238, 0.04415243, 0.05487315, -0.00902092, 0.003410176, 0.007892423, -0.011299073, -0.030130861, 0.016193926, 0.01199733, 0.053377893, 0.015150067, -0.024601227, 0.012469888, 0.024008766, 0.022894375, -0.01281549, 0.005550793, 0.01383819, -0.032867465, 0.01229356, -0.011433082, 0.054224264, 0.039525595, -0.010198789, 0.03221858, 0.021356799, -0.032528915, -0.038961347, 0.019833328, -0.0004454476, -0.0016010546, -0.009479372, -0.02194926, -0.007412812, 0.024699971, 0.03735324, 0.0014317802, 0.02403698, -0.02165303, 0.031936456, 0.06325224, 0.018168796, 0.02399466, -0.013034136, 0.006005718, -0.013048243, -0.016941557, 0.009394734, -0.030864384, 0.0026219913, -0.0010218185, 0.011186223, 0.0009830263, 0.0018620194, -0.0074833436, 0.006633444, 0.006809772, 0.003949738, -0.011482453, 0.0038157292, 0.022908483, -0.006936728, -0.024855139, -0.003295563, -0.024051085, 0.013309208, 0.0038827339, -0.008872805, -0.016941557, -0.023416305, 0.011581196, -0.057525117, 0.025461705, 0.0017650393, 0.03890492, -0.022837952, -0.012914234, -0.0011232068, 0.019057486, -0.012942445, -0.01570021, -0.03258534, 0.008033485, -0.001395633, 0.021046462, 0.024685863, 0.03738145, 0.007991167, 0.023670217, -0.021892834, 0.00060965266, 0.017167255, -0.0072717504, 0.017491698, -0.022555826, 0.028593283, 0.025729723, -0.0006775388, 0.03419345, 0.03549122, -0.0044751945, 0.0190857, 0.010650187, 0.02217496, -0.00420365, 0.050754137, 0.013718287, 0.029961586, 0.029566612, -0.019720478, -0.026576096, 0.020707913, -0.008908071, 0.0034489678, -0.022301914, 0.022837952, -0.011686994, 0.023529155, 0.018408602, -0.025165476, 0.03506803, -0.011228542, -0.039892357, 0.0076878835, -0.04784826, 0.011637622, -0.010805355, -0.0018884686, 0.003456021, 0.034390934, -0.0017491698, 0.00963454, -0.00881638, 0.0101494165, -0.017364742, 0.015629679, -0.031823605, -0.011016948, -0.011877427, 0.004626836, -0.018874107, 0.0091126105, -0.032980315, -0.006569966, -0.069628246, -0.017463485, -0.009380628, -0.004337659, -0.01610929, 0.011947958, -0.0021952786, 0.022598146, -0.021215737, 0.023599686, 0.022203172, -0.018718937, 0.03535016, 0.001702443, 0.0074833436, 0.017689185, 0.008597733, 0.004133119, -0.005769439, -0.0025638032, 0.05208012, 0.025151368, 0.0075820866, -0.020270621, 0.0050359163, -0.023218818, 0.008541309, -0.018027734, 0.02358558, -0.011404869, 0.00016365405, 0.015559147, 0.0209054, -0.028000822, -6.58129e-06, 0.0046479953, 0.02154018, 0.029679462, 0.01945246, -0.008167494, -0.0058470233, 0.040343754, -0.009973089, -0.021046462, 0.009987195, -0.0044998806, -0.017364742, 0.00086665014, 0.037437875, -0.02020009, -0.0040343753, 0.034701273, -0.01577074, 0.017759716, -0.004605677, 0.014839731, 0.019692266, -0.018380389, 0.02883309, -0.009676859, -0.02801493, 0.0022940221, -0.01774561, -0.010241107, 0.003928579, 0.014726881, -0.01354196, 0.004968912, 0.020820763, -0.02106057, 0.016983874, 0.03046941, 0.007652618, -0.018352177, 0.004439929, 0.016715856, 0.014219058, -0.004669155, 0.03221858, -0.03732503, 0.008809327, -0.0042953403, -0.0031915295, -0.057835452, -0.0014873233, 0.016320882, 0.024319103, -0.048356082, -0.010621974, 0.011955012, -0.038735647, 0.02551813, -0.0013312733, 0.021243948, -0.006661657, 0.015516829, 0.0012907181, 0.0023310508, -0.013344473, -0.007991167, -0.013922827, 0.0065029617, 0.009627487, -0.011877427, 0.0061044614, -0.041274764, -0.01845092, 0.007843052, -0.008075804, 0.007060157, 0.030271923, -0.009458212, 0.013570172, 0.020961825, -0.023162393, -0.008047592, -0.05594522, -0.028818984, 0.009274832, -0.012342932, -0.0015128908, 0.032387853, -0.024417846, -0.018380389, 0.034165237, 0.0065029617, 0.018775363, 0.0031280515, 0.01640552, 0.022301914, 0.061277367, -0.02217496, -0.04485774, 0.0039955834, 0.0062702093, 0.030864384, -0.012258295, -0.007546821, -0.012914234, -0.002837111, 0.015093642, 0.017124936, -0.017618654, 0.00922546, 0.018958744, -0.025250113, -0.009084398, -0.0343063, 0.04694546, -0.0005100276, 0.0370429, 0.045506626, -0.010734824, -0.0021564865, 0.023938235, 0.025151368, 0.010163523, -0.013104667, 0.0056319037, 0.025391174, -0.0038686276, -0.041641526, -0.00024333209, 0.012131339, -0.010276373, 0.027408361, -0.011637622, 0.017872566, -0.03456021, 0.020990038, 0.008541309, -0.0017200757, 0.012603897, -0.010911152, 0.011602356, 0.004178964, 0.006171466, 0.010071833, -0.001363894, -0.004944226, -0.011249701, -0.0001786419, -0.020439895, -0.008491937, 0.02190694, -0.038143188, -0.003621769, 0.023486836, 0.014169686, 0.0060586166, -0.036450442, 0.0005029745, -0.031061871, 0.024714077, 0.00563543, -0.061954465, -0.024530696, 0.004344712, 0.008484884, 0.054280687, -0.011715205, -0.01004362, 0.0077725207, -0.02481282, 0.039553806, 0.0050429692, -0.00094335264, -0.012864862, 0.022203172, 0.020143665, -0.018323964, -0.02987695, -0.0508952, -0.009295991, -0.007173007, -0.006312528, 0.042572536, 0.0016918633, -0.013379739, 0.01804184, -0.021441435, 0.013753553, 0.038679224, 0.058794674, 0.0067180814, -0.00012673547, -0.007102476, -0.025123157, -0.013492588, -0.017703291, 0.00849899, 0.0019396036, -0.032669976, 0.004439929, -0.02242887, -0.012441675, -0.028127778, 0.0051593455, 0.028522752, 0.025602767, -0.006682816, 0.015643785, 0.0009160219, -0.008738796, -0.006139727, 0.018775363, -0.004312973, 0.058174003, -0.016899237, -0.061164517, 0.03396775, 0.008308557, 0.01792899, 0.002727788, 0.002688996, -0.018775363, 0.013337419, -0.021258056, -0.019819222, 0.001858493, -0.009655699, 0.027478894, 0.013894615, -0.020425789, 0.0018567296, 0.008879858, -0.010904099, -0.026646627, -0.01763276, 0.016871024, 0.020806657, 0.033939537, -0.008618893, -0.030328346, 0.0028141886, -0.005811758, 0.016236246, -0.005875236, 0.013563119, 0.013767659, -0.0044258228, -0.03221858, 0.0054308902, -0.015756635, -0.018380389, -0.032077517, -0.016250351, -0.00697552, -0.0011540642, -0.021356799, 0.018549664, 0.00064668147, -0.035829768, 0.011609409, 0.0032109255, -0.017096724, -0.013499641, 0.024601227, 0.0061820457, 0.012272401, 0.008922176, -0.0038827339, -0.01640552, -0.0023575, 0.0105373375, -0.021229843, 0.013711235, 0.01949478, -0.012780225, 0.012476941, 0.0010932311, 0.011150957, -0.007864211, -0.021850517, -0.013048243, 0.027916186, -0.0090491325, -0.0029711202, -0.0057659126, -0.012230082, -0.022626357, -0.056199133, -0.032190368, -0.02440374, 0.030328346, 0.03035656, -0.011341391, 0.011489507, 0.021709453, -0.008456672, -0.017167255, -0.007786627, -0.012117233, -0.0023557367, -0.035632282, 0.022739207, -0.0012492811, 0.019762797, -0.0032250318, 0.01129202, -0.019339612, -0.024756394, -0.038002126, -0.0004756437, 0.036450442, -0.039666656, 0.020510426, -0.015291129, -0.015220598, -0.031682543, 0.02429089, -0.026251653, 0.04717116, -0.010008355, -0.014712775, 0.0104527, -0.004210703, -0.0190857, 0.025137262, 0.017943097, 0.0016089893, -0.029989798, 0.014077996, 0.027493, -0.028226523, 0.001658361, -0.04274181, 0.021963365, -0.0015508012, 0.0148820495, -0.030751534, 0.030977232, -0.0016160425, 0.022894375, -0.020468108, 0.038002126, 0.017237786, 0.0033449347, -0.03819961, 0.0020524533, 0.010276373, 0.014698668, -0.031456843, -0.0147550935, -0.037861064, -0.0331778, -0.006559387, -0.0073634405, -0.021526074, 0.02551813, -0.008104016, 0.00038461457, 0.016349096, 0.009437053, 0.0021635396, 0.03128757, 0.019438354, -0.022231383, -0.002177646, 0.012956552, 0.0053286203, -0.023049545, -0.026787689, 0.0025549869, 0.0032655871, 0.036027256, 0.004841956, -0.0016151608, 0.005416784, -0.008901018, 0.024714077, 0.0020365838, -0.013055296, -0.02388181, 0.013943987, 0.041782588, 0.011877427, -0.012695587, -0.027013388, -0.022273703, -0.008809327, 0.029707674, 0.007028418, -0.036055468, 0.01711083, 0.02533475, -0.0012783752, -0.017914884, -0.013041189, 0.016786387, -0.009592221, -0.009274832, 0.017449379, 0.0015358134, 0.025786148, 0.014296642, 0.010156469, 0.04017448, -0.020595064, 0.0007758415, 0.007927689, -0.023571474, -0.02213264, -0.029397337, 0.016828706, -0.005501421, 0.00038395336, 0.008971549, -0.010001302, 0.038453523, -0.012857809, 0.026533777, 0.0050888145, 0.0046338895, 0.044660255, -0.01536166, -0.0038792072, 0.01845092, 0.03738145, -0.03797391, 0.013090561, 0.027958505, 0.035321943, -0.011595303, -0.017209575, -0.017449379, -0.029735887, -0.003953265, 0.034249872, 0.0010835332, -0.02008724, -0.014303695, -0.035942618, 0.0003012998, -0.038030338, 0.02510905, -0.03072332, -0.014811519, -0.007596193, 0.007236485, 0.02649146, 0.0052616154, -0.016391413, -0.015291129, 0.0056319037, 0.005335673, -0.0038016231, 0.022866163, -0.02206211, -0.02826884, -0.040118057, -0.0044751945, 0.024333209, 0.0066263913, 0.01240641, 0.042572536, -0.029340914, -0.049089603, -0.009923717, -0.028353479, -0.013965146, 0.02767638, 0.032754615, -0.03456021, -0.027196769, -0.018422708, -0.020312939, -0.010629028, 0.014176739, -0.0009689202, 0.014444757, -0.02488335, 0.0011478927, -0.0004452272, 0.008308557, -0.032303218, -0.026801795, 0.018352177, 0.013083508], "content_hash": "004024ec2fb45632cde7582d544365b0"}, {"content": "# OPT Overview\n\n## What Is OPT?\n\nOPT (Open Pre-trained Transformer) is a family of language models released by Meta in 2022. OPT was designed to replicate GPT-3's architecture and performance while being openly available to researchers. It uses a decoder-only transformer architecture similar to GPT-2 but with options for much larger sizes.\n\n## Architecture Details\n\nOPT's architecture is close to GPT-2 but has some differences:\n\n| Property | OPT-125M | OPT-350M | OPT-1.3B |\n|----------|----------|----------|----------|\n| Parameters | 125M | 350M | 1.3B |\n| Layers | 12 | 24 | 24 |\n| Attention Heads | 12 | 16 | 32 |\n| Hidden Dimension | 768 | 1024 | 2048 |\n| Vocabulary Size | 50,272 | 50,272 | 50,272 |\n\n### Key Differences from GPT-2\n\n- **Learned positional embeddings**: Like GPT-2, OPT uses learned absolute position embeddings (unlike LLaMA's RoPE)\n- **LayerNorm placement**: OPT uses pre-norm LayerNorm (applied before each sublayer), which is slightly different from GPT-2's original arrangement\n- **Larger variants available**: OPT scales up to 175 billion parameters, though only smaller variants are practical for interactive use\n\n### Similarities to GPT-2\n\n- Same general decoder-only architecture\n- Same tokenizer style (BPE with ~50K vocabulary)\n- Same attention mechanism (standard multi-head self-attention)\n- Similar training objective (next-token prediction)\n\n## What to Expect in the Dashboard\n\nWhen using OPT models:", "source_file": "opt_overview.md", "chunk_index": 0, "embedding": [-0.00838912, 0.031035196, 0.023107445, -0.010371058, -0.013529162, -0.0016765244, -0.016115429, 0.026382515, -0.037273426, -0.001244397, 0.007329921, -0.037195448, -0.04026258, -0.03472615, 0.019442484, -0.036207728, 0.0022093733, -0.008590563, -0.025290826, 0.05869135, 0.027630161, -0.014919767, -0.0384171, 0.00013676588, -0.031243136, -0.012067077, -0.022249687, 0.05520834, 0.021586876, 0.010955892, -0.0042238017, -0.024420073, -0.030229425, -0.04267989, 0.020612152, -0.022990478, 0.031217145, 0.013399199, 0.0051952763, 0.04473331, -0.021599872, -0.027630161, -0.048138343, 0.042030077, -0.03423229, 0.041536216, 0.032204863, -0.030489352, -0.04026258, 0.028072037, -0.023341378, 0.0786017, -0.049334005, 0.008655544, -0.011462748, -0.013477176, -0.045149192, -0.012963822, 8.3562234e-05, 0.0149067715, 0.0098707, -0.0067905737, 0.040860407, -0.00032165876, 0.04286184, -0.011716177, -0.07927751, 0.029865528, 0.011677188, 0.015777525, 0.016999178, 0.0016684018, -0.016271384, -0.017116144, 0.011040368, 0.0008277027, -0.00976673, 0.0627462, 0.00064412976, -0.051803306, 0.0068295626, -0.0010535136, -0.018961621, -0.04611092, 0.010358062, 0.00023312136, -0.006420179, -0.039638754, -0.027292257, -0.022444634, -0.018636713, -0.015179694, -0.02770814, 0.015972469, -0.0065338966, -0.018168846, 0.0041003367, -0.020885075, -0.0026382515, 0.038755007, -0.0076418323, 0.0028705606, 0.024822958, 0.02245763, -0.0034895102, -0.0031743494, 0.011833143, -0.022925496, -0.020300241, -0.0058710845, -0.10698565, 2.4659998e-05, -0.01786993, 0.008194176, 0.0047663976, -0.019377504, 0.014399915, 0.020300241, 0.010462032, -0.061914437, -0.046864707, 0.0106439805, 0.044317428, 0.026122589, 0.022860516, 0.008486592, 0.010267087, -0.009038935, -0.002946914, -0.0074078985, -0.006108267, 0.030619314, 0.019195555, -0.046552792, -0.02262658, -0.0034895102, -0.063889876, 0.016648278, -0.028202, -0.013620136, 0.01825982, -0.05770363, 0.009610774, 0.028825823, -0.094353236, -0.020703128, 0.014815797, 0.001235462, -0.084372066, 0.06420179, 0.005481195, 0.006306461, -0.027448213, -0.030541336, -0.014334934, -0.021690847, 0.014542875, 0.041172322, 0.02879983, -0.009935682, 0.038261145, 0.0041685672, -0.016297378, 0.026902368, 0.010488025, 0.03028141, -0.04135427, -0.013067793, -0.059627086, -0.057755616, 0.04777445, 0.016232396, -0.0095198, -0.0043212743, 0.0058678356, 0.0013881687, -0.035869826, -0.012203538, -0.021872794, -0.030437365, -0.013412195, 0.0143479295, -0.01811686, 0.04624088, -0.01906559, -0.0066086254, -0.04592897, 0.035375964, 0.0088959765, 0.04403151, -0.0067450865, -0.0027373484, 0.064981565, 0.011846139, -0.015374638, -0.041328277, 0.062330317, 0.010208604, 0.0014986374, -0.004733907, 0.006264223, 0.0069660237, 0.038936954, -0.057391718, 0.0338424, -0.0046786726, -0.018350795, -0.0033303052, -0.03514203, 0.0038144179, 0.013048299, 0.0056176563, 0.018246824, 0.005679389, -0.0034310266, -0.056352016, -0.059835024, 0.028721852, 0.026174575, 0.030021483, -0.00539347, -0.0008756266, 0.031685013, 0.024563031, -0.00050848076, 0.03415431, 0.01805188, -0.011189826, 0.0020387967, -0.05879532, -0.020820094, -0.039456807, -0.04691669, -0.0040873406, -0.011670689, -0.030437365, 0.024030183, -0.02143092, 0.0040645967, -0.054168634, 0.016102431, 0.024225127, -0.029163728, -0.02312044, 0.025719704, 0.004022359, -0.039144896, -0.076054424, 0.0056404, 0.012508951, 0.013067793, -0.0086490465, -0.02657746, -0.031139167, -0.0073949024, 0.021145001, -0.013028804, 0.031918947, -0.053284883, 0.023575312, 0.011846139, 0.010695966, -0.022639578, 0.025043895, -0.035895817, 0.0229125, -0.016128425, 0.0009942179, -0.063266054, -0.04808636, 0.03761133, -0.02636952, -0.023601305, 0.0032117139, -0.05305095, -0.0074014002, -0.0071869614, 0.005809352, -0.02083309, 0.023224412, -0.04561706, -0.02227568, 0.07283134, 0.030125454, 0.008057714, -0.021937776, 0.00234421, -0.007888762, -0.0045032226, -0.011488741, 0.011748667, -0.04434342, 0.040522505, 0.0069075404, 0.010897408, 0.020365223, 0.026772406, -0.022990478, 0.036779568, -0.015764527, 0.023627298, -0.050425693, -0.008584064, -0.02178182, 0.03236082, 0.009318356, 0.010669973, 0.030697292, -0.0077652973, -0.018636713, -0.002862438, -0.010163117, 0.029241705, -0.0009162401, -0.018558735, 0.03581784, 0.016726255, -0.009422327, 0.001367862, -0.02217171, -0.056975838, -0.04223802, -0.006306461, -0.0018406028, -0.012430973, 0.025394795, -0.03028141, -4.9319995e-05, -0.012534944, -0.011553722, 0.045565072, -0.053128928, -0.0012793245, 0.0014645221, -0.011462748, 0.03615574, 0.011599209, 0.031918947, 0.019975333, 0.003973623, -0.033582475, -0.020729119, -0.02672042, -0.0033173088, -0.06799671, 0.021989763, -0.039326843, 0.028253986, 0.054168634, 0.023575312, -0.000611639, -0.04254993, 0.022002758, -0.00024063486, 0.032438796, 0.020820094, 0.0139970295, -0.013106782, -0.049723893, 0.007999231, -0.049230035, 0.0117941545, 0.0021232727, -0.006894544, -0.007934249, -0.032308836, 0.011605708, 0.033660453, -0.026148582, 0.014490889, 0.047072645, 0.027292257, -0.03870302, 0.015075724, -0.045383126, -0.0016383477, -0.012489457, -0.0011924118, -0.019910352, 0.03615574, 0.016323369, 0.011644697, -0.009916187, -0.0077068135, -0.026642442, 0.023484338, 0.013763095, -0.023315385, -0.04720261, 0.009539294, -0.0072844336, 0.014062011, -0.049152058, -0.0034895102, -0.007849773, 0.02809803, -0.018844655, 0.029215712, 0.010656977, 0.043121766, 0.00057183777, -0.017999893, -0.041224305, -0.0046429327, -0.019611437, -0.023029467, 0.06056282, 0.0051237964, 0.020807097, 0.023328383, 0.059523113, 0.008766013, 0.01949447, -0.051933266, 0.037871256, -0.003522001, 0.02111901, 0.024835955, 0.020040315, 0.047670476, 0.0007013948, 0.000119911296, -0.06259024, -0.00034602685, -0.077769935, 0.012281516, 0.040626474, 0.013880063, -0.023809247, -0.016973184, -0.0137501, 0.034206297, 0.040730447, -0.019728404, -0.024186138, -0.014568867, 0.009929183, 0.03810519, 0.010150121, 0.0058970773, 0.031685013, -0.011040368, 0.0001301662, -0.046266876, -0.0257327, -0.007589847, -0.014477893, -0.028981779, -0.0046396838, -0.047462534, 0.018870646, 0.054896425, 0.009396334, -0.009214386, -0.006540395, -0.05749569, 0.044057503, -0.02260059, 0.08021325, -0.00059011386, 0.032516774, -0.039560776, -0.0037559345, -0.06019892, -0.00025992625, 0.017506033, 0.023029467, -0.01308079, -0.03288067, 0.014412912, 0.03472615, -0.04065247, -0.027734132, 0.002982654, 0.024913933, -0.019325517, 0.03623372, -0.012229531, -0.0018162348, -0.041120335, 0.050139777, 0.024511047, -0.017986897, 0.0049515953, 0.020495186, 0.12715593, -0.019130573, 0.010832427, -0.0081616845, -0.016947193, -0.03584383, -0.010708962, 0.037039492, -0.014113996, 0.013568151, -0.017077155, -0.019052595, -0.04709864, -0.019260537, 0.00856457, 0.043615628, -0.026252553, -0.04226401, 0.06763282, 0.028046044, 0.040210593, -0.039430816, -0.004889863, -0.010150121, 0.018597724, -0.030203432, 0.000539347, -0.007511869, 0.0014970128, -0.0013995405, 0.03969074, 0.007979737, 0.029839534, 0.0037786781, 0.035505928, -0.006644365, 0.007927751, -0.027396228, -0.0037007001, 0.0026658687, -0.020378219, -0.05650797, -0.11311991, -0.029969499, 0.022028752, 0.011976102, -0.038339123, 0.03584383, -0.007375408, -0.061030686, -0.00796674, 0.0057118796, -0.010812933, -0.045409117, 0.000150676, 0.0120410845, 0.010546508, 0.010825929, 0.019780388, -0.0068685515, -0.0009609149, 0.013015808, 0.0133342175, -0.002094031, 0.036623612, 0.02616158, -0.013828077, 0.015322654, -0.012749383, -0.017635997, -0.026382515, -0.0016156042, -0.020248257, 0.025511762, -0.009545792, 0.006231732, -0.0055364296, 0.001431219, -0.00613426, 0.011787656, 0.029267697, 0.0069075404, -0.025329815, 0.015205687, -0.01152773, -0.01226852, -0.021274965, 0.017324086, 0.016323369, -0.0037266929, 0.0048736176, 0.019455481, -0.0015408754, 0.0110273715, 0.0032588255, -0.01889664, -0.027578177, 0.034960084, 0.040626474, -0.025849666, 0.030905234, -0.011573217, 0.017077155, -0.01875368, 0.044083495, 0.03373843, -0.025108878, -0.004357014, -0.0058743334, -0.01755802, -0.02227568, 0.052193195, -0.021157999, -0.017038167, 0.004204307, -0.0007606905, -0.0062284833, -0.013984033, -0.01293783, -0.004058099, 0.008220168, -0.00771981, 0.015946476, -0.0077068135, 0.0025960135, -0.0005909261, -0.011787656, 0.0009081174, -0.0005413777, -0.053804737, 0.0046266876, -0.025069889, -0.016583296, 0.02929369, 0.026772406, 0.0074014002, -0.024446065, -0.005890579, 0.032932658, -0.013275734, 0.0012119062, -0.008343633, -0.022093732, -0.0131587675, 0.036051773, 0.004486977, -0.0038793995, 0.045461103, 0.017973902, 0.0040808422, 0.015361642, 0.007388404, -0.018064875, 0.04522717, -0.012339999, -0.039820705, 0.0048378776, -0.027838103, -0.011618704, 0.010143623, -0.03553192, -0.022444634, 0.036779568, -0.021248972, 0.0049580936, -0.0088959765, -0.058431424, 0.04008063, 0.012242527, 0.016206402, -0.0074338913, 0.0019900606, -0.014685834, -0.023081452, 0.024731984, -0.07205156, -0.028747845, 0.004945097, -0.041744158, 0.010689467, 0.04262791, -0.00087806344, 0.02118399, 0.03285468, -0.03327056, -0.058015544, 0.006176498, 0.028747845, 0.015595576, 0.014204971, -0.015426624, 0.0148028005, -0.039560776, -0.00895446, 0.01970241, -0.0169212, 0.008772511, -0.056975838, -0.012788372, 0.0068295626, -0.0006201678, 0.013100284, -0.012385487, 0.014075007, 0.017051162, -0.006475413, -0.029189719, 0.0010600118, -0.027396228, -0.0072129536, -0.027188288, 0.021417923, 0.038625043, -0.003937883, -0.04712463, -0.013282232, 0.02552476, -0.048372276, -0.0028331962, -0.011300295, 0.008350131, -0.0119371135, 0.044863272, 0.010760947, -0.021209983, -0.032074902, -0.009922685, -0.014075007, 0.0048671192, 0.014256956, 0.011638198, 0.0048671192, -0.010247593, 0.01885765, -0.003479763, -0.015517598, 0.019624433, -0.026642442, -0.009506803, 0.019104581, -0.0029582859, -0.012677903, 0.0138930585, -0.07293531, 0.035479937, 0.030203432, -0.0049970825, -0.03254277, -0.03976872, 0.00831764, 0.001150986, -0.0049483464, 0.0003791268, -0.00934435, 0.0045941966, 0.010604992, 0.018831657, -0.0059685567, 0.01582951, 0.006420179, -0.0002416502, 0.0019510716, 0.029527623, 0.02622656, -0.03236082, -0.014386918, 0.01078694, -0.0075768507, -0.0041133333, -0.022210699, 0.00493535, 0.034284275, 0.010527014, -0.026135586, -0.00047802064, 0.0027438467, 0.0009235505, -0.010981885, 0.017246109, -0.041224305, -0.0029387914, -0.012716892, -0.019546455, 0.010325571, 0.015270668, 0.029345674, -0.0015904239, 0.017934913, -0.014049015, 0.008259157, -0.014685834, 0.050607644, -0.020287246, -0.027838103, -0.00063316413, -0.02295149, -0.021768825, 0.021495903, -0.008668541, 1.1815984e-05, -0.010514017, -0.014101, -0.009110415, -0.005939315, -0.017506033, -0.0275002, 0.009311859, 0.023432354, 0.0038339123, -0.0031840967, 0.011118346, -0.012034586, -0.08369625, -0.016011458, 0.016713258, -0.022717556, 0.0034342757, -0.01885765, 0.038027212, -0.014386918, -0.00044431144, -0.041718166, 0.03059332, -0.075534575, 0.0057443706, -0.000777342, 0.032594755, -0.059003264, 0.017233111, -0.024822958, 0.0253818, -0.0023523327, -0.024731984, 0.010553006, 0.0054649496, -0.01564756, -0.053700767, -0.008934965, 0.022704558, 0.026343526, -0.0043960027, -0.011222317, -0.057859585, 0.05375275, -0.002633378, 0.037585337, -0.008934965, -0.017921915, -0.0035804843, -0.027994059, 0.01336021, -0.00035577407, 0.012398483, -0.019871363, 0.02802005, -0.03475214, 0.023341378, 0.0050003314, 0.029449645, 0.016986182, -0.019013606, -0.0012037836, 0.025537755, 0.020079304, 0.011989099, 0.011488741, 0.016336367, 0.0014409663, 0.020742116, -0.006680105, 0.051543377, 0.006144007, 0.021002043, 0.05008779, 0.020768108, -0.004402501, -0.023510331, -0.004282285, 0.02939766, 0.034076333, -0.0131587675, -0.02499191, 0.003817667, 0.05442856, -0.001648095, -0.013529162, -0.014867783, -0.031217145, 0.01642734, 0.018103864, -0.014516882, -0.024030183, 0.046864707, -0.0039541284, 0.010215102, 0.029735565, 0.026668435, 0.034648173, -0.0067450865, -0.015140705, 0.017389067, -0.008428109, 0.009539294, -0.0032084647, -0.004993833, -0.004142575, -0.030541336, -0.005854839, 0.008843991, 0.030957218, 0.037923243, -0.020794101, 0.050139777, 0.021937776, -0.034908097, -0.016830226, -0.0036194732, -0.006465666, -0.017830942, -0.013958041, -0.0131587675, 0.0037786781, 0.00278446, 0.023692278, 0.014932764, 0.035869826, -0.0032848183, 0.016947193, 0.086607434, 0.0035609899, 0.03670159, -0.006049784, 0.0050425697, 0.015764527, -0.021651858, 0.0165703, -0.034362253, -0.015283665, 0.003528499, 0.024615018, -0.002428686, -0.0049873353, 0.005175782, 0.00093248545, -0.008629552, 0.0003387164, -0.03217887, 0.03256876, 0.021443916, -0.000203372, -0.008187678, 0.03979471, -0.012034586, 6.3712396e-05, 0.0049970825, -0.014230963, 0.035271995, -0.007661327, -0.024589024, -0.044187464, -0.042082064, 0.009890194, 0.056248043, 0.027136302, -0.016297378, 0.049126063, 0.012671405, 0.014049015, -0.028202, -0.017506033, -0.055988118, 0.015725538, 0.004896361, 0.010884413, 0.01906559, -0.0042985305, 0.032022916, -0.029917512, -0.025290826, 0.029475639, 0.022834523, -0.014412912, -0.0060302895, 0.019260537, 0.018246824, 0.0257327, 0.030827254, 0.033712436, -0.0077328063, 0.02682439, -0.0035837335, -0.0013930423, 0.020495186, 0.027630161, -0.0007680009, 0.027864095, 0.021937776, -0.0076158396, -0.02929369, 0.0062512266, -0.022184707, 0.030983211, 0.011885128, 0.026694428, 0.012346498, -0.005679389, -0.029033763, -0.029605601, 0.016024454, -0.028487919, -0.039638754, -0.0049970825, -0.021482905, 0.017142138, -0.02295149, -0.025394795, -0.0067905737, 0.017986897, -0.0056533962, 0.020144286, 0.0047761453, -0.00451297, 0.010124128, 0.029371668, 0.0030525092, 0.01807787, -0.0012720141, -0.0027649656, 0.00078830763, -0.02474498, -0.029969499, 0.014152985, -0.060302895, -0.021755828, -0.0019721906, -0.02908575, 0.0021557636, 0.009571785, 0.010877914, 0.0097342385, -0.0069010425, 0.014958756, -0.031243136, -0.0048248814, 0.015608572, 0.002558649, 0.02280853, 0.025355807, -0.0056371507, -0.0018406028, 0.01906559, 0.024030183, 0.010312575, 0.011813649, -0.007219452, -5.5285098e-05, 0.005598162, -0.00077003153, 0.012417978, 0.023926213, -0.0006104206, -0.02227568, -0.0020615403, 0.028227992, 0.0136851175, -0.010812933, 0.035246, 0.023393365, 0.027890088, 0.03553192, 0.023419356, 0.01963743, -0.02760417, 0.06877649, 0.023952205, -0.023562316, 0.015400631, 0.0014458399, 0.014165982, -0.0052570086, 0.042601913, -0.008805002, -0.03553192, 0.04494125, -0.0103645595, 0.022301674, 0.02460202, -0.004522717, 0.028487919, 0.0012240902, 0.013347214, 0.008454102, -0.011228815, -0.004279036, 0.007128478, -0.0039996156, -0.010507519, -0.020456197, -0.007095987, -0.024264116, 0.0053284885, 0.021976765, -0.0052472614, 0.06576134, 0.0043830066, -0.029709572, -0.0002745471, 0.009643264, 0.0052082725, 0.010247593, 0.012781874, -0.015140705, 0.023588309, -0.0016164165, -0.006066029, -0.028565897, 0.003937883, 0.0105725005, 0.0012378988, -0.0676848, 0.0036389676, -0.01906559, -0.012242527, -0.018324802, 0.019507466, 0.010273586, 0.021989763, 0.012307509, -0.00058077276, -0.02309445, 0.010604992, -0.0064851604, -0.00042928447, -0.0015498104, -0.01385407, -0.009578283, 0.0017008925, -0.019598441, 0.028981779, 0.01195011, 0.01582951, 0.004184813, 0.024718987, 0.029033763, -0.0012630792, 0.00613426, -0.04286184, -0.008902474, -0.004311527, -0.01706416, -0.008785508, -0.022535607, 0.018272817, 0.008967456, -0.016453333, -0.020053312, 0.014217966, 0.0027405976, 0.00814219, 0.021885792, 0.04452537, 0.018675702, 0.05432459, -0.026486486, -0.039456807, -0.009155903, -0.022236692, 0.026395513, -0.015452617, -0.004636435, -0.022938494, -0.014919767, 0.010449036, 0.0461889, -0.0039996156, -0.014984749, 0.043381695, 0.003908641, -0.033998355, 0.01628438, 0.0468907, 0.0078822635, -0.008200673, 0.0012525197, -0.03680556, 0.014334934, -0.005185529, 0.018675702, -6.9994794e-06, -0.02978755, 0.023211416, 0.01790892, 0.0007168279, -0.048372276, 0.010078641, 0.028150015, 0.018272817, -0.0007643457, 0.022366654, 0.019780388, -0.03405034, -0.008629552, -0.008538578, -0.017739968, 0.003908641, -0.033582475, -0.0034602685, 0.0060790256, 5.3051357e-05, -0.018948624, 0.0013434939, -0.0039931173, -0.009461316, 0.017311089, -0.039196882, -0.02210673, 0.019871363, -0.023523327, 0.022340663, 0.01875368, 0.012236029, -0.0015132582, -0.034804128, 0.033374533, -0.036467656, 0.017103149, -0.024796966, -0.043485664, -0.006797072, 0.01504973, -0.008012227, 0.006001048, -0.014711826, 0.0165703, -0.0044317427, -0.01385407, 0.019429488, 0.0026723668, 0.0014661466, 0.0033368035, 0.00771981, 0.011423759, -0.011274301, 0.0032458291, -0.009324855, 0.025433784, -0.022938494, 0.00080374075, 0.0042855344, -0.0370135, -0.040834416, 0.028513912, 0.009792722, 0.018636713, 0.024290109, 0.04262791, -0.014217966, 0.028253986, -0.03236082, -0.029605601, 0.008908973, 0.007843275, 0.024641009, -0.0019640678, -0.02097605, 0.039196882, -0.0087920055, -0.023523327, -0.020716123, -0.0106764715, 0.03197093, 0.0005397531, -0.017233111, -0.0031516058, 0.033790413, 0.007128478, 0.0017496287, 0.0135421585, 0.027786117, 0.029215712, 0.014672837, -0.030021483, 0.027136302, -0.012476461, 0.009526297, -0.014958756, 0.006972522, -0.0040873406, -0.0057963557, -0.01529666, -0.026954353, 0.0061992416, 0.020742116, -0.0039541284, 0.011105349, -0.043485664, 0.018233828, 0.03740339, 0.027344244, -0.021067023, -0.02616158, -0.008200673, 0.033010636, 0.02340636, 0.00064128684, 0.038235154, 0.0032766955, -0.030437365, 0.010130626, -0.053856723, -0.0190396, -0.01385407, -0.033790413, -0.030021483, -0.013477176, 0.012476461, 0.011020874, -0.03722144, -0.003645466, -0.02672042, -0.024524042, -0.011430258, 0.016830226, 0.009350847, -0.024589024, 0.03394637, 0.008343633, -0.009636766, -0.00035963237, -0.01720712, 0.013867066, 0.014269952, 0.012008593, -0.03444023, 0.0116576925, 0.033218578, 0.011475745, -0.007563854, -0.0038858976, 0.011488741, -0.042731877, -0.007661327, -0.007810784, 0.021378934, -0.0098707, -0.0061310106, 0.005607909, 0.02672042, 0.014334934, 0.0014084755, 0.004984086, 0.033972364, 0.004857372, 0.0035804843, 0.010475028, -0.006472164, -0.0057898574, 0.010780442, -0.0029582859, -0.028539903, 0.029943505, -0.02340636, -0.035194017, 0.0036422168, -0.012580431, -0.022964485, 0.00796674, 0.041094344, 0.007206456, 0.00094142044, 0.014191974, 0.013945044, 0.0019543206, -0.029267697, -0.016336367, -0.0124244755, 0.055156354, -0.023042463, -0.01311328, 0.014958756, 0.002587891, -0.0044089993, -0.004282285, -0.023978198, 0.05177731, 0.007804286, -0.020937061, 0.018441768, -0.029189719, 0.00035516487, 0.0016586544, 0.010124128, -0.009337851, -0.029839534, 0.028539903, -0.019338515, -0.0070894887, -0.014516882, 0.01350317, 0.00079561805, 0.0014287822, 0.026174575, -0.040938385, 0.013802085, -0.004980837, 0.019117577, -0.0023848235, 0.01836379, 0.014594859, 0.02227568, -0.04611092, 0.023497334, -0.00072170154, 0.0007545984, -0.021521894, -0.029501632, -0.023419356, 0.013022305, -0.05858738, 0.020222263, -0.005013328, 0.0040743444, -0.018909635, -0.035297986, 0.022925496, 0.009851205, -0.013529162, 0.05770363, 0.0027763373, -0.029449645, -0.0074858763, 0.004600695, -0.017623, -0.024069171, -0.013477176, -0.0035090046, -0.0014888901, 0.045591068, 0.00038095444, 0.015959473, 0.0022337413, 0.006631369, 0.012996313, 0.005149789, -0.02552476, -0.010702464, 0.015179694, 0.029241705, 0.030437365, 0.010026656, 0.0025294074, -0.008558072, -0.026798397, 0.028383948, -0.015816513, -0.014893775, 0.0094548175, 0.026850384, -0.0066606104, -0.035168022, -0.04611092, 0.07631435, -0.02752619, -0.013763095, 0.011781158, 0.0019429488, 0.015504601, 0.014659842, -0.025394795, 0.009220884, -0.014438904, -0.014269952, -0.0027227276, -0.01770098, -0.02569371, 0.0031711005, -0.017947908, -0.021093016, -0.0017057661, 0.03158104, 0.014828794, 0.02616158, -0.027188288, 0.02488794, 0.010929899, 0.017947908, 0.085151844, 0.014776808, 0.04028857, 0.018935628, 0.032048907, -0.0070764925, 0.0053187413, 0.0038793995, 0.03493409, 0.029241705, -0.0023929463, -0.0045389626, -0.022327665, 0.008538578, 0.03444023, -0.008610058, -0.031918947, -0.012755881, -0.00040694705, -0.018168846, 0.0009714744, 0.02548577, 0.0072129536, -0.03813118, -0.007108983, 0.02379625, 0.008642549, 0.049230035, -0.016232396, -0.0036324696, 0.00472416, 0.036259715, 0.025537755, 0.008343633, -0.009396334, -0.0377153, -0.025563749, -0.0086490465, -0.0010031529, -0.0056371507, -0.018818662, -0.014750816, 0.005299247, -0.026460493, -0.019169562, -0.019026602, -0.0010380805, 0.023770256, 0.023003474, -0.038547065, -0.01350317, 0.00047274088, -0.0010112756, 0.0007521616, 0.0165703, -0.002600887, 0.028617881, -0.032802694, -0.022574596, 0.0063649444, 0.0077263084, 0.001099813, -0.010026656, 0.019091584, -0.021664854], "content_hash": "3c69810f4c7559fe34f9d1ea00f0b527"}, {"content": "though only smaller variants are practical for interactive use ### Similarities to GPT-2 - Same general decoder-only architecture - Same tokenizer style (BPE with ~50K vocabulary) - Same attention mechanism (standard multi-head self-attention) - Similar training objective (next-token prediction) ## What to Expect in the Dashboard When using OPT models:\n\n- **OPT-125M is very similar to GPT-2**: Same number of layers (12), heads (12), and hidden dimension (768). You'll see similar attention patterns and predictions.\n- **Different module paths**: The dashboard auto-detects OPT's internal structure (e.g., `model.decoder.layers.N.self_attn`), so hooking works automatically.\n- **Tokenization**: OPT's tokenizer is very similar to GPT-2's, so the same text usually produces similar (but not identical) token sequences.\n- **Good for comparison**: Running the same prompt on GPT-2 and OPT-125M can show how similar architectures with different training data produce different predictions.\n\n## HuggingFace Model IDs\n\n- `facebook/opt-125m`\n- `facebook/opt-350m`\n- `facebook/opt-1.3b`\n- `facebook/opt-2.7b`\n\nNote: OPT models are not in the default dropdown but can be loaded by typing the model ID directly.", "source_file": "opt_overview.md", "chunk_index": 1, "embedding": [-0.004660339, 0.012660587, -0.001214439, 0.0053076083, -0.01627235, 0.0026586587, 0.013735054, 0.054940216, 0.0059063323, -0.019676987, 0.023353476, -0.045774885, -0.018731974, -0.04903712, -0.008899952, -0.018977936, -0.0010898396, 0.0018447174, -0.021528177, 0.04631859, -0.00032505055, 0.018796701, 0.01242757, 0.018796701, -0.023923073, 0.010641107, -0.030784126, 0.022188392, 0.015016647, 0.017683396, -0.0048868833, -0.028453957, -0.0065989103, -0.042046614, 0.036091734, -0.02653804, 0.014472941, 0.015275555, -0.00037582073, 0.061412908, -0.02014302, -0.02684873, -0.046370372, 0.024389107, -0.03192332, 0.021618795, 0.04895945, -0.009139442, -0.055665158, 0.020155966, -0.028583411, 0.060221933, -0.05276539, 0.019625206, -0.023612384, -0.008401555, -0.05939343, -0.034098145, 0.014926029, 0.032829497, -0.015832206, -0.016738383, 0.021683522, 0.0072947247, 0.046551608, -0.015482682, -0.052972518, 0.012744732, 0.002252497, 0.02718531, 0.02098447, -0.006100513, 0.010065038, -0.026279133, 0.010291582, -0.013126621, 0.032415245, 0.04160647, 0.035548028, -0.07037111, 0.025411792, 0.0047833202, -0.039716445, -0.061930727, -0.015728643, -0.014084579, -0.011955064, -0.042150177, -0.024648014, -0.032052774, -0.007819013, -0.010777034, -0.04160647, -0.0021990973, -0.0072623612, -0.020194802, -0.007223525, -0.045412414, -0.0004652248, 0.0398459, 0.0014401742, 0.010039147, 0.0011553756, 0.028945882, 0.0012492298, 0.0080131935, 0.025217611, -0.006809273, -0.02923068, 0.009586059, -0.096779704, 0.008142647, -0.022188392, 0.021515232, -0.01098416, -0.0169714, -0.019366298, 0.0073788697, -0.014239924, -0.06607325, -0.047483675, 0.015210828, 0.028790537, 0.01795525, 0.011942118, 0.002699113, -0.0013786836, -0.023806565, -0.007760759, -0.029101226, -0.014097525, 0.013230184, 0.012434043, -0.0011545665, 0.008815807, -0.023638275, -0.06793738, 0.01132074, 0.003203983, -0.016686602, 0.009404823, -0.063691296, 0.0039742333, 0.024583288, -0.064001985, 0.0083497735, -0.0038253614, -0.0030955654, -0.08854644, 0.029489588, -0.0046376847, -0.01596166, -0.0011804574, -0.0325447, -0.031379614, -0.0446098, -0.005136082, 0.03953521, 0.010958269, -0.029256571, 0.03953521, -0.0093983505, 0.005857787, 0.02569659, -0.021722358, 0.069024794, -0.05097893, -0.007941994, -0.038370125, -0.044325, 0.046448044, 0.03624708, -0.014291706, 0.002362533, -0.011411358, -0.0009126497, -0.04870054, -0.023275804, -0.038111217, -0.038007654, -0.010938851, -0.029929731, 0.025709536, 0.04929603, -0.016337076, -0.019858222, -0.04660339, 0.0037056166, 0.015456791, 0.027004074, 0.0029774387, 0.009579586, 0.05405993, 0.012472879, -0.014472941, -0.04253854, 0.04326348, 0.022770934, 0.011942118, 0.010272164, -0.011773828, -0.0059063323, 0.02651215, -0.041347563, 0.025062267, -0.014343487, -0.018977936, 0.004382013, -0.03490076, 0.0023512058, 0.010188019, -0.008420973, 0.04590434, 0.031767976, -0.04191716, -0.033528548, -0.055509813, 0.011980955, 0.034848977, 0.06819629, -0.044066094, 0.029929731, 0.04497227, 0.009942057, -0.0033560912, 0.0060001863, 0.0255283, -0.0020712616, -0.007922576, -0.04119222, -0.028453957, -0.028402176, -0.033114295, 0.014835412, -0.012841823, -0.018887319, 0.016738383, -0.028117377, -0.025243502, -0.07503146, 0.012589388, 0.04023426, -0.02213661, -0.02923068, 0.02569659, -0.0044726306, -0.029852059, -0.08419679, 0.005061646, 0.0014862921, 0.014705958, -0.011378994, -0.024945758, -0.024194926, -0.005977532, 0.017968195, -0.018641356, 0.03288128, -0.05204045, 0.008026139, -0.01343731, 0.035781045, -0.02418198, 0.019016773, -0.040130697, -0.0043981946, -0.036713112, -0.015883988, -0.07000864, -0.057322167, 2.6699858e-05, -0.018058812, -0.0089582065, 0.014511777, -0.06457158, -0.024233762, -0.030421656, 0.003381982, -0.040648513, 0.011832083, -0.038421907, 0.0028803484, 0.090048105, 0.004689466, -0.003495254, -0.012673533, 0.010518126, 0.0008632954, -0.025437683, -0.01857663, -0.0064888746, -0.058875613, 0.021178652, -0.029593151, 0.017916413, 0.022045992, 0.02788436, -0.011650847, 0.033709783, -0.016000496, 0.015508573, -0.047897927, -0.0071329074, -0.046448044, 0.02160585, 0.003485545, 0.024583288, 0.074876115, -0.019715823, -0.031094817, -0.040104806, -0.031819757, 0.012815932, -0.0036311806, 0.006725128, 0.022602644, 0.012453461, 0.00838861, 0.011268958, -0.03624708, -0.015676862, -0.034434725, -0.02502343, 0.015456791, -0.0045114667, 0.029437806, -0.022486135, -0.0015898552, -0.022602644, -0.011722047, 0.036739003, -0.041062765, 0.002140843, 0.037774637, -0.018511903, 0.0053852806, -0.012032736, 0.03627297, -0.006194367, 0.0272112, -0.04626681, 0.0033528549, -0.0339428, -0.0058545507, -0.07503146, -0.0022023336, -0.022926278, 0.027055856, 0.026952293, 0.015974605, -0.014434105, -0.03055111, 0.0029094755, -0.007074653, 0.035832826, 0.02485514, 0.01512021, -0.0035340902, -0.030628782, 0.012032736, -0.034408834, 0.030110966, -0.0018722264, -0.010440454, 0.018110594, -0.0051716818, 0.004763902, 0.022343736, -0.03425349, -0.0031085107, 0.027470108, 0.040726185, -0.037800528, -0.00461503, -0.06198251, -0.014498832, -0.02432438, 0.019521642, -0.0076377774, 0.032389354, 0.020518437, -0.0014693013, 0.003867434, -0.024945758, -0.021411669, 0.008103811, 0.021683522, -0.034512397, -0.080623865, 0.013968071, 0.0048998287, -0.0015987551, -0.03220812, -0.02163174, -0.022188392, 0.01814943, 0.010900015, 0.02177414, -0.0013875836, 0.057218604, -0.005802769, -0.016427694, -0.042331412, 0.0065924376, -0.02098447, 0.0027508945, 0.028091487, -0.012201026, 0.015417955, 0.04556776, 0.060273714, -0.008738135, 0.02718531, -0.027936142, 0.010835288, -0.011165395, 0.019741714, 0.013307856, 0.016712492, 0.042357303, -0.008375664, -0.023936018, -0.054215275, -0.020363092, -0.074927896, -0.01764456, 0.06581434, 0.004676521, -0.0040680873, -0.018615466, -0.009832021, 0.03668722, 0.048571087, -0.029463697, -0.037101477, -0.038991503, 0.02920479, 0.041994832, 0.03363211, 0.0021602612, 0.013579709, -0.013993962, 0.014006907, -0.030732345, -0.03748984, -0.027806688, -0.033839237, -0.049166575, -0.034124035, -0.04556776, 0.0021554066, 0.049140684, -0.009786712, 0.012084518, -0.016375912, -0.048778214, 0.05877205, -0.030706454, 0.0706818, -0.010097401, 0.0356257, -0.05139318, 0.0032590008, -0.027237091, -0.008375664, 0.043418825, 0.026097897, -0.009967947, -0.004734775, -0.014835412, 0.042512648, -0.050745912, -0.005867496, -0.01343731, 0.009599004, -0.02261559, 0.038292453, 0.019676987, -0.03358033, -0.030421656, 0.040078916, 0.022550862, -0.002466096, 0.006530947, 0.030110966, 0.09372459, -0.0075342143, -0.0026602768, 0.0030146567, -0.02379362, -0.045464195, -0.019055609, 0.028169159, -0.0140198525, 0.012168663, -0.007974357, 0.015586245, -0.061568253, -0.023974854, -0.018977936, 0.02264148, -0.0074047605, -0.057995327, 0.08217731, 0.020492546, 0.019664042, -0.014291706, 0.0010073128, -0.05369746, -0.012563497, -0.023767728, 0.0036797258, -0.016168786, 0.0019790258, -0.002716913, 0.028428067, -0.010168601, 0.027133528, 0.009728458, 0.04590434, -0.009883802, 0.031146599, -0.026952293, 0.0038253614, -0.028686974, -0.015547409, -0.04730244, -0.11381583, -0.024803359, 0.0031732377, 0.023094568, -0.0342276, 0.03117249, -0.040648513, -0.049140684, 0.0076766135, -0.0047412477, -0.00017769565, -0.056493662, 0.0061231675, 0.015236719, -0.008446864, 0.017631615, 0.015275555, -0.007948467, 0.014201088, -0.0018172085, 0.033166077, -0.013061894, 0.034512397, 0.03329553, 0.00070390536, 0.019534588, -0.011948591, -0.022939224, -0.019016773, -0.017916413, -0.029153008, 0.019340407, -0.022550862, 0.007372397, 0.010498708, 0.017579833, -0.0048415745, 0.0064435657, 0.036739003, 0.026408587, -0.017864631, 0.004026015, -0.009573113, -0.009547222, -0.017696341, 0.018434228, -0.005239645, -0.01644064, 0.017903468, -0.0072817793, 0.0112042315, 0.0056474246, -0.0053011356, -0.024764523, -0.021049198, 0.01829183, 0.03293306, -0.024893977, 0.007540687, 0.002877112, 0.017566888, -0.028402176, 0.033347312, 0.047483675, -0.02362533, -0.0062008398, -0.010531072, -0.013398474, -0.006521238, 0.035729263, -0.011385467, -0.0272112, -0.018783756, 0.014744794, -0.0025518592, -0.0025421502, -0.008899952, -0.011832083, 0.013217239, 0.012408152, 0.030758236, -0.01629824, 0.009967947, 0.0034046364, -0.0077025043, 0.011935646, 0.0072817793, -0.06659106, 0.0153014455, -0.02196832, -0.02028542, 0.032363463, 0.026641604, -0.001494383, -0.022240173, 0.01941808, 0.03264826, -0.007741341, -0.0051231366, 0.0060293134, -0.017489215, -0.004213723, 0.007838431, 0.006139349, -0.006126404, 0.049865626, -0.0023657694, 0.0013341838, -0.008932316, -0.002644095, -0.0051037185, 0.024246708, -0.02331464, -0.043082245, 0.011197759, 0.006352948, -0.010349836, -0.00528819, -0.015948715, -0.010641107, 0.035056103, -0.021230433, -0.0117932465, 0.00025263728, -0.053127863, 0.033865128, 0.022084828, 0.0014620195, -0.01817532, 0.017942304, -0.022071883, -0.024699796, 0.00915886, -0.038059436, 0.0009304496, -0.0013196203, -0.056493662, 0.0068998905, 0.032725934, -0.00373798, 0.014032798, 0.011184813, -0.05030577, -0.09170511, 0.003485545, 0.03264826, 0.01632413, 0.028479848, -0.011437248, -0.001176412, -0.04701764, -0.018265938, 0.01581926, -0.014576504, 0.00997442, -0.06203429, -0.007870794, 0.023638275, -0.0011634666, 0.009320678, -0.01666071, 0.012544079, 0.012446988, 0.0006715419, -0.047897927, 0.013890399, -0.0063400026, -0.02379362, -0.029075336, 0.009055297, 0.02956726, -0.009728458, -0.033735674, -0.022149555, 0.035832826, -0.031457286, -0.023573548, -0.0062752757, -0.0086216265, -0.00070673716, 0.046111465, -0.002093916, -0.0070422897, -0.047716692, -0.0061425855, -0.013696218, 0.0095666405, 0.007204107, 0.010997105, 0.009003515, 0.0029353662, 0.027832579, -0.0057315696, -0.010971215, 0.030007403, -0.018045867, -0.0075083235, 0.017372707, 0.001559919, -0.032000992, -0.0047380114, -0.04523118, 0.031845648, 0.0058254236, -0.008304465, -0.02367711, -0.025036376, -0.0023010422, -0.003139256, 0.013670327, -0.010964742, -0.012116881, 0.006239676, 0.011637902, 0.020699672, -0.031483177, -0.01629824, -0.004958083, 0.0032088375, -0.0066733463, 0.029774386, 0.0071523255, -0.032363463, 0.004178123, 0.01074467, 0.0037347437, 0.0003903843, -0.032777715, -0.009527804, 0.04598201, 0.029075336, -0.026926402, 0.027910251, 0.010919433, 0.0019952075, -0.014175197, 0.030887691, -0.026563931, -0.0006266376, 0.007035817, -0.01975466, 0.0063367663, 0.027288873, 0.016375912, -0.036169406, 0.008589263, -0.0053399717, 0.04126989, -0.042072505, 0.026292078, 0.012505243, -0.011217177, -0.010356309, -0.028065596, -0.013514983, 0.02328875, -0.020570219, -0.0021926246, 9.845573e-05, 0.00528819, -0.0011076395, -0.019223899, -0.024842195, -0.007560105, 0.025929607, -0.007158798, -0.02126927, 0.0055729887, 0.003673253, -0.009942057, -0.10480584, -0.0065374197, 0.016117005, -0.01694551, -0.0030567292, 0.0036667804, 0.04455802, 0.011016523, -0.0014264197, -0.033658, 0.017942304, -0.07166565, -1.6409287e-05, -0.0010315854, 0.0068610543, -0.0563901, 0.026227351, -0.0339428, 0.017087908, -0.015146101, -0.01576748, 0.02328875, 0.01317193, -0.0084145, -0.090099886, -0.014576504, 0.0023641512, 0.018537793, 0.010291582, -0.029282462, -0.051237836, 0.02853163, -0.009437187, 0.0125699695, 0.018615466, 0.015702752, -0.006660401, -0.022706207, -0.002428878, 0.0005214563, -0.001448265, -0.024712741, 0.01310073, -0.05206634, 0.029463697, 0.0107640885, 0.009955002, 0.008110284, -0.01141783, 0.0047930293, 0.013618546, 0.01394218, 0.028143268, 0.023042787, 0.008369192, -0.012019791, 0.030447546, -0.0052720085, 0.04626681, 0.0051490273, 0.036169406, 0.02297806, -0.0036279443, 0.011618484, -0.0073982878, 0.012621751, 0.027107637, 0.030343983, -0.012291644, -0.015909879, -0.00076175504, 0.045774885, 0.00069338724, -0.013812726, 0.00087138626, -0.016065223, 0.0055826977, 0.01961226, -0.022939224, -0.007961412, 0.051108383, -0.02266737, 0.00029996887, 0.03855136, 0.0010234945, 0.03565159, 0.016479475, -0.02328875, 0.01579337, -0.02143756, 0.021398723, -0.0030971835, -0.024091363, 0.0054661892, -0.04155469, -0.011553757, -0.010880597, 0.043108135, 0.047768474, -0.016893728, 0.04119222, -0.002716913, -0.036790784, -0.006210549, -0.02536001, 0.004362595, -0.017903468, -0.023612384, -0.009353042, -0.0012735024, -0.02822094, 0.030732345, 0.014641231, 0.030784126, -0.018214157, 0.022693262, 0.06011837, -0.006110222, 0.018382447, -0.0065730195, 0.021592904, 0.024868086, -0.0023609148, 0.01562508, -0.009508386, -0.01579337, -0.009365987, 0.025541246, -0.008116757, 0.007922576, -0.011495503, 0.014498832, 0.00056878786, -0.009320678, -0.026162624, 0.018421283, 0.0093142055, -0.004828629, -0.025735427, 0.038059436, -0.01961226, 0.019676987, -0.014097525, -0.014201088, 0.036480095, 0.0076766135, -0.037463948, -0.038111217, -0.0031149834, 0.004148996, 0.043444715, 0.010336891, -0.0119615365, 0.0446098, 0.017437434, 0.023042787, -0.0130748395, -0.018408338, -0.039716445, 0.020712618, -0.013747999, 0.04129578, 0.026952293, -0.003027602, 0.04391075, -0.032156337, -0.0070293443, 0.027107637, 0.02098447, 0.0132431295, -0.004430558, 0.032700043, 0.026356805, 0.0119615365, 0.041684143, 0.039794117, -0.008123229, 0.0028932937, -0.009670204, 0.021955375, 0.019353352, 0.027107637, 0.008938788, 0.0030696746, 0.027495999, -0.017748123, -0.01644064, 0.021191597, -0.025968444, 0.026563931, 0.022848606, 0.02064789, -0.0006921736, 0.0015121829, -0.03497843, -0.0189391, 0.02211072, -0.028246831, -0.035496246, -0.014900139, -0.0054758983, -0.013333747, -0.01132074, -0.024427943, -0.021579958, 0.029023554, 0.0009733312, 0.006977563, 0.0054953163, -0.011870919, -0.007081126, 0.049917407, 0.0027233856, 0.01958637, -0.0036700168, 0.008285047, -0.011437248, 0.0029547843, -0.037800528, 0.005210518, -0.042797446, -0.009942057, 0.0028981483, -0.037386276, -0.015249664, 0.010466345, 0.012026263, 0.004385249, 0.0032282555, 0.00915886, -0.013476146, -0.009689622, 0.009463077, 0.021178652, 0.022266064, 0.010524599, -0.0056765517, 0.0009425859, 0.026136734, 0.04525707, -0.0012411389, 0.03153496, -0.004475867, 0.0076377774, 0.03627297, 0.005333499, 0.018900264, 0.024246708, 0.0127382595, -0.011029469, 0.0153014455, 0.022524972, 0.01750216, -0.021113925, 0.031094817, 0.008194429, 0.013307856, 0.056907915, 0.01266706, 0.00822032, -0.0030534929, 0.057166822, 0.011980955, -0.020194802, 0.029644933, -0.011042414, 0.018524848, -0.009243005, 0.033036623, -0.015314391, -0.02920479, 0.045075834, -0.038758487, 0.009793185, 0.021929484, 0.009929111, 0.02227901, 0.00082931377, 0.031094817, 0.0034369999, -0.005754224, -0.0025421502, 0.0032460555, 0.0088740615, 0.013359638, -0.011210704, -0.021372832, -0.014628286, 0.021567013, 0.011171868, 0.007941994, 0.04598201, 0.016673656, -0.013540873, -0.0013107203, -0.0015429282, 0.01209099, 0.011870919, 0.010770561, -0.024971649, 0.009676676, 0.007663668, 0.0077866497, -0.055872284, 0.007437124, 0.016621875, 0.04323759, -0.040441386, -0.0051037185, -0.030343983, -0.027729016, -0.011968009, 0.01783874, -0.008213847, 0.019651096, 0.024389107, 0.017126745, -0.02182592, 0.0125699695, -0.027262982, -0.016311185, -0.012466406, -0.0029758206, 0.008278574, 0.0034499452, -0.032337572, 0.035470355, -0.012925968, -0.009068242, -0.012887131, 0.020000622, 0.021838866, 0.0013463201, 0.02819505, -0.052247576, -0.0143564325, -0.01927568, -0.0046279756, -0.008110284, -0.030266311, 0.007204107, 0.020376038, -0.009217114, 0.001747627, 0.039975353, 0.0033884547, 0.0070487624, 0.018188266, 0.03818889, 0.0067316005, 0.052584156, -0.025139939, -0.01377389, -0.008550427, -0.023042787, 0.0105116535, 0.007242943, -0.0014401742, -0.039690554, -0.0034369999, 0.017346816, 0.022175446, 0.0069646174, -0.00788374, 0.022563808, 0.0067316005, -0.012815932, -0.0018932626, 0.036195297, 0.0029483116, -0.010421036, -0.019715823, -0.018719029, 0.007935521, 0.0103822, 0.019625206, 0.00889348, -0.017023182, 0.011281904, 0.025981389, 0.010861179, -0.04727655, 0.011845028, 0.015366172, 0.014252869, 0.014680067, 0.0027913488, 0.017994085, -0.054888435, -0.018641356, 0.0020243346, -0.0050195735, 0.015249664, -0.038758487, 0.012246335, -0.020337202, 0.012680005, -0.019482806, 0.0054402985, -0.0011925937, -0.0063367663, 0.025230557, -0.019120336, 0.0012718842, 0.020013567, -0.031146599, -0.01148903, 0.018550739, 0.02025953, 0.004780084, -0.02126927, 0.015728643, -0.017929358, -0.0033107824, -0.0031554378, -0.055250905, -0.011592593, 0.011404885, -0.011145977, 0.016505366, -0.033450875, 0.005268772, -0.025567137, -0.032078665, 0.013048949, -0.0022411698, 0.0066183284, 0.0034628906, 0.0100067835, 0.0068739997, -0.013372583, -0.0058221873, -0.0038059433, 0.036894348, -0.017398598, -0.011670265, 0.022045992, -0.019236844, -0.013644436, 0.01767045, 0.032492917, 0.020855017, 0.016738383, 0.033787455, -0.006809273, 0.025813099, -0.0080131935, -0.03117249, 0.0053108446, -0.009100606, 0.017269144, 0.0005085109, -0.0029483116, 0.041166328, 0.013618546, -0.013890399, -0.033166077, -0.010181546, 0.033347312, 0.014434105, -0.015236719, -0.004469394, 0.034382943, 0.0075730504, 0.018162375, 0.005278481, 0.024673905, 0.040363714, 0.0010647579, -0.018900264, 0.017346816, -0.015612136, 0.0043787765, -0.00041708414, 0.027573671, 0.0062008398, 0.0044143763, -0.0026651314, -0.023612384, -0.002924039, 0.003792998, 0.012039209, 0.011605538, -0.040104806, 0.011980955, 0.034874868, 0.008375664, -0.035729263, -0.022421408, -0.00088352256, 0.0448946, 0.029515479, -0.016285295, 0.014201088, 0.004204014, -0.027599562, 0.008304465, -0.026900511, -0.0067963274, 0.002867403, -0.024220817, -0.040001243, 0.0102398, 0.0005457289, 0.003812416, -0.02165763, 0.0036926712, -0.030680563, -0.019664042, -0.023211077, 0.005576225, 0.012142772, -0.017204417, 0.030421656, -0.005941932, 0.004938665, -0.019236844, -0.018447174, -0.0056571336, 0.009767294, 0.021243379, -0.030007403, -0.0035179085, 0.03593639, 0.01293244, -0.013010113, -0.0050584096, 0.01978055, -0.047509566, 0.024052527, -0.0047218297, 0.015159046, -0.015456791, -0.01014271, 0.0028981483, 0.016893728, 0.024622124, -0.0013438929, -0.0053076083, 0.027159419, 0.003110129, 0.0067898547, 0.012802986, -0.007585996, 0.0064047296, 0.01610406, -0.0074435966, -0.019987676, 0.032829497, -0.026693385, -0.040985093, -0.0071911616, -0.02334053, -0.013722109, -0.008187956, 0.015068429, -0.0071134893, -0.01141783, 0.023612384, 0.026382696, -0.0031004199, -0.006323821, -0.0023188423, -0.016557148, 0.054888435, -0.019392189, 0.00532379, 0.0038965612, 0.016065223, -0.0027055857, -0.009417769, -0.017256198, 0.055820502, 0.024505615, -0.017256198, 0.0087899165, -0.00027630306, -0.0019871166, 0.026771057, 0.017566888, 0.0062558576, -0.023366421, 0.011314267, -0.024453834, -0.023133405, -0.012524661, -0.014887193, 0.0035114358, 0.010000311, 0.02129516, -0.030343983, 0.025088158, -0.020997416, 0.025139939, -0.004618266, 0.026745167, 0.01156023, 0.02819505, -0.041347563, 0.03425349, -0.003792998, 0.017683396, -0.030706454, -0.03492665, -0.01593577, 0.017929358, -0.03355444, 0.024091363, -0.015223773, 0.016479475, -0.025282338, -0.036894348, 0.023016896, -0.0028172396, -0.008945261, 0.0342276, 0.0006060059, -0.02415609, -0.012311062, -0.01646653, -0.020673782, -0.009812603, -0.00047817017, -0.019133281, 0.0061976034, 0.046033792, -0.008641045, 0.03500432, -0.005783351, -0.0074500693, 0.03557392, -0.008505118, -0.017851686, -0.021282215, 0.013476146, 0.018252993, 0.020194802, 0.032441135, 0.0033658003, -0.0014426014, -0.033088405, 0.00908766, -0.016065223, -0.010919433, -0.0073853424, 0.035366792, 0.01576748, -0.021191597, -0.03288128, 0.06695353, -0.0143564325, -0.021812975, 0.0032153102, -0.0064435657, 0.02617557, 0.0053658625, -0.016375912, 0.03156085, -0.0062720394, -0.01276415, 0.015741589, -0.035107885, -0.019198008, 0.013502037, 0.0048804106, 0.0051296093, 0.003915979, 0.043755405, -0.0039289244, 0.03692024, -0.028324503, 0.017036127, 0.019456916, -0.0032120738, 0.035159666, 0.016285295, 0.013398474, 0.023599438, 0.013514983, -0.032337572, 0.018408338, 0.005372335, 0.03259648, 0.0055147344, 0.021722358, -0.017269144, -0.026771057, 0.011806192, 0.03565159, -0.011864446, -0.030654673, -0.038033545, -0.00048221563, -0.036091734, -0.0068998905, 0.04391075, 0.02569659, -0.045748994, -0.0067963274, 0.008071448, -0.010265691, 0.042667992, -0.014900139, -0.00882228, 0.0027152947, 0.022732098, 0.034124035, 0.022524972, 0.00031372334, -0.04259032, -0.029696714, -0.039431646, 0.006352948, -0.0077672317, -0.012336953, 0.012343425, -0.013463201, -0.031146599, -0.009417769, -0.009741403, -0.027029965, 0.026978184, 0.022020102, -0.034149926, -0.009236532, -0.002941839, -0.014472941, -0.005576225, 0.029800277, 0.002877112, 0.014252869, -0.0462927, -0.016919618, 0.00020490118, -0.022175446, -0.0058318963, 0.0025178776, 0.030085076, -0.010673471], "content_hash": "1c7ebbbf92b955e60843eb6fcf8145a6"}, {"content": "# Output and Prediction\n\n## How Does the Model Choose the Next Token?\n\nAfter your text has passed through all the Transformer layers (attention + MLP in each), the model needs to make a prediction: what token comes next? This final step converts the model's internal representation into a probability distribution over its entire vocabulary.\n\n## Logits: Raw Scores\n\nThe model's final hidden state for the last token is multiplied by the embedding table (in reverse) to produce a score for **every token in the vocabulary**. These raw scores are called **logits**. A higher logit means the model thinks that token is more likely.\n\nFor GPT-2, this means producing about 50,257 scores -- one for each token in its vocabulary.\n\n## Softmax: Turning Scores into Probabilities\n\nRaw logits can be any number (positive or negative). To get actual probabilities, the model applies a function called **softmax**, which:\n\n1. Converts all scores to positive numbers\n2. Makes them all add up to 1.0 (100%)\n3. Preserves the ranking (higher logits \u2192 higher probabilities)\n\nAfter softmax, we can say things like \"the model predicts 'mat' with 45% probability.\"\n\n## Temperature\n\n**Temperature** is a setting that controls how \"confident\" or \"creative\" the model's predictions are:\n\n- **Low temperature (e.g., 0.1)**: Makes the model very confident -- the top prediction gets almost all the probability. Good for factual, predictable text.\n- **High temperature (e.g., 1.5)**: Spreads probability more evenly, making less likely tokens more probable. Good for creative, varied text.\n- **Temperature = 1.0**: The default, unmodified distribution.\n\n## Greedy Decoding vs. Sampling\n\nOnce we have probabilities, how do we pick the actual next token?", "source_file": "output_and_prediction.md", "chunk_index": 0, "embedding": [0.01852944, 0.012449853, 0.026149021, -0.02459047, 0.027039621, -0.012975555, 0.01397748, 0.019222127, 0.0033119207, 0.027237533, -0.00091302017, -0.0018724258, -0.061204046, -0.0011356702, 0.00074757874, -0.05046736, 0.01987771, 0.032210052, -0.03404073, 0.01758936, -0.03676201, -0.037405223, -0.01464543, 0.03997807, 0.042748824, -0.0168843, -0.009982147, 0.036910444, 0.0055786227, -0.01330953, -0.010439817, -0.0219063, -0.03973068, -0.0455938, 0.006323882, -0.013173466, -0.0076566907, 0.039087467, -0.0069330777, 0.01918502, -0.031022584, -0.009901746, -0.072732374, 0.026198499, -0.009870823, 0.019506626, 0.0067104273, -0.047721345, -0.0387906, 0.048983026, -0.017181167, 0.033422258, -0.041685052, 0.017972814, 0.015981331, -0.020731201, -0.0315421, 0.008411227, -0.012901338, -0.045470104, -0.004295292, 0.009914115, 0.044282638, 0.025678981, 0.04621227, 0.012214834, -0.07030796, 0.04801821, -0.028400261, 0.0036087874, 0.042056136, -0.00625585, -0.0120664, -0.0062094647, -0.020904373, -0.010817085, 0.0025558379, 0.009611064, -0.016315307, -0.034906592, 0.0010583611, -0.038666904, 0.0058229193, -0.03943381, -0.0053745266, -0.020026142, 0.02482549, -0.03940907, -0.0410171, -0.07421671, -0.029958813, -0.005999184, 0.005071475, 0.03134419, 0.016723499, -0.025357375, -0.0015987516, -0.035030287, -0.004533404, -0.019061325, -0.000112388065, -0.019989034, 0.014163022, -0.013569288, 0.021139393, -0.031220496, -0.044554766, -0.00889982, -0.031517364, -0.043441515, -0.080599345, -0.008621507, -0.04123975, 0.05294125, 0.010223351, -0.061995693, -0.000660606, 0.0306515, 0.006002276, -0.032581136, -0.011299494, -0.044084724, 0.02029827, 0.049477804, -0.01624109, 0.008287532, 0.026074804, -0.044356853, 0.027806528, 0.028400261, 0.0038314376, 0.004814809, 0.013247683, -0.029167166, -0.021015698, -0.03129471, 0.00052802096, 0.059472322, -0.011559253, 0.041437663, 0.0026687093, -0.03223479, -0.01650085, 0.015461815, -0.062342037, -0.04081919, -0.0055105905, 0.019234497, -0.07857075, 0.027806528, 0.008485444, -0.00463236, -0.035772454, 0.021832082, 0.016166873, -0.036143538, 0.015845267, 0.08030248, 0.023848303, -0.03948329, -0.016661651, 0.03473342, 0.01582053, 0.001048311, -0.022252643, 0.053040206, -0.024887336, 0.009487369, -0.019086063, -0.059224933, 0.011565438, 0.009641987, -0.020397225, -0.016439002, -0.026990144, -0.02346485, -0.07159439, -0.042080875, -0.018987108, -0.03804843, -0.03671253, 0.010909856, -0.04054706, 0.00014640405, 0.0012508607, -0.0020224054, -0.028944517, 0.026693277, 0.009561586, -0.009376044, 0.0077742003, -0.0018585101, 0.07629478, -0.023167983, -0.041437663, -0.00053575187, 0.06338107, 0.0069825556, 0.018603655, 0.027559139, 0.011497405, 0.077977024, 0.00891219, -0.044530027, 0.023477219, 0.017688315, -0.013012663, 0.021844452, -0.03250692, 0.003562402, -0.0038283453, -0.041536618, 0.0056868554, -0.026816972, -0.054969843, 0.019927187, -0.040398628, 0.03559928, 0.031245235, 0.07960979, 0.016265828, 0.04935411, -0.0073536388, 0.0028820822, -0.049552023, 0.012876599, 0.033743862, -0.017106952, 0.04373838, 0.020558028, 0.013495072, 0.015152578, -0.013260052, -0.049972583, -0.020125099, -0.029117689, 0.021052806, 0.0150412535, -0.008256609, -0.03319961, -0.019494256, 0.016377155, 0.015313381, 0.023031918, 0.023798825, -0.0044097095, -0.028721867, -0.043392036, 0.0455938, 0.0293156, 0.023749348, -0.059670236, -0.019246867, 0.0059837224, -0.011132507, -0.021498106, -0.026643798, 0.037479438, 0.0015229888, -0.027089098, 0.0055848076, 0.009475, -0.03837004, -0.006221834, -0.016797716, -0.02523368, -0.034782898, -0.03629197, -0.027113838, -0.060857702, -0.016599804, -0.029661944, -0.03809791, 0.045915402, -0.053633943, -0.04626175, -0.00067258894, 0.033545952, -0.05318864, 0.04054706, -0.03861743, -0.044332113, 0.047498696, 0.029711423, 0.032135837, -0.005776534, -0.009679096, 0.030033028, 0.0055786227, -0.020854896, 0.04210561, -0.011089213, -0.0114850355, -0.03592089, 0.02773231, 0.053040206, 0.024738902, 0.03881534, -0.01467017, -0.018813936, 0.030181462, -0.06847728, 0.0051054913, -0.008083437, 0.019692168, 0.024219386, -0.008466889, 0.033446997, 0.015313381, -0.005838381, -0.008825604, -0.0123323435, -0.01444752, 0.025604764, 0.00066137913, 0.029859856, 0.024924444, -0.037949476, 0.018900521, -0.024664687, -0.012827122, -0.027410705, 0.025456332, 0.018034661, -0.0015925668, 0.032779045, -0.022252643, 0.0052384627, 0.007965927, -0.016859563, 0.019444779, 0.00027116158, -0.0121653555, 0.040002808, -0.05279282, -0.0051085837, 0.01690904, 0.040942885, -0.031764753, 0.03587141, -0.051704306, -0.008392673, -0.027138576, -0.001066092, -0.040324412, 0.03814739, 0.018987108, 0.024169909, 0.038864817, 0.01329716, 0.0073660086, 0.011744794, 0.00889982, -0.014732017, -0.0056652087, 0.030206202, 0.009543031, 0.0065063317, -0.024949184, -0.032383226, -0.056849997, 0.017341971, 0.060412403, -0.00024275048, 0.0050312746, -0.028400261, -0.03290274, 0.012524069, 0.011225278, -0.011014997, -0.0052786632, 0.056652088, -0.021164132, 0.001260911, -0.026915926, -0.016995626, -0.006518701, 0.02142389, -0.008862712, 0.0383453, 0.016785346, 0.03837004, -0.007130989, -0.002606862, 0.02795496, 0.011305679, 0.018863413, -0.03161632, -0.017626468, -0.0048550097, -0.010186244, -0.019098433, -0.04888407, 0.02592637, 0.0008140645, 0.052396998, -0.027237533, 0.045766972, 0.029389817, 0.044727936, -0.023044288, -0.016105026, -0.047300782, -0.026149021, 0.010798532, 0.0016745145, 0.021065176, 0.03537663, 0.042575654, 0.0193953, 0.09024752, 0.011274755, 0.02570372, -0.038691644, 0.015375229, -0.0020502366, 0.05501932, 0.038172126, 0.023167983, 0.0029763994, -0.032135837, -0.025654243, -0.03629197, -0.017725425, -0.05214961, -0.019457147, 0.03067624, -0.02366276, 0.020669354, -0.02595111, 0.0018028476, 0.009475, 0.0024398742, -0.042550914, 0.030602023, -0.0136929825, 0.033348043, 0.03723205, -0.011559253, -0.04754817, 0.014855712, -0.008534921, -0.021300195, -0.011590176, -0.023774086, -0.0038809155, -0.0232422, -0.047028653, -0.010922226, -0.024862597, 0.00070273946, 0.020459073, -0.015103101, 0.026767492, -0.00021433941, -0.021238348, 0.07431567, 0.02435545, 0.06288629, -0.051308487, 0.00056976784, -0.027039621, -0.021300195, -0.055167753, -0.015931854, 0.0100439945, 0.028177612, -0.0058507505, 0.010328492, 0.028969256, -0.0012686419, 0.014546475, -0.008479259, -0.044059988, 0.016946148, 0.013952741, 0.048760377, 0.0025712997, 0.021263087, -0.026445888, 0.033966515, 0.013272421, -0.04777082, 0.0029980459, -0.038988512, 0.07075326, 0.023477219, -0.012864229, 0.0041375817, -0.041116055, -0.018677872, -0.048760377, 0.022438185, -0.012245757, 0.023477219, -0.0042334446, -0.008887451, -0.039557505, 0.003519109, 0.0037819599, 0.02953825, -0.011590176, -0.019321084, 0.057889033, 0.009048254, 0.045692753, 0.05976919, 0.021559954, -0.040349152, 0.036663055, -0.064222194, -0.011008812, -0.019568473, 0.0026903558, -0.010854194, 0.0035747716, -0.0012941539, 0.028375521, 0.009066808, -0.0038932848, -0.007836048, 0.01963032, -0.002071883, 0.014398041, 0.021807343, -0.031072062, -0.019308714, 0.008343195, -0.047053393, -0.022401078, 0.049081985, -0.023774086, 0.018752089, 0.02523368, -0.025629504, 0.003630434, -0.014608323, -0.04354047, -0.0455938, -0.0040169796, -0.015461815, 0.0043385853, 0.005727056, 0.012554993, 0.006797014, 0.0019018032, 0.014051697, -0.024380188, -0.024677055, -0.0032902742, -0.009765682, -0.025777938, 0.019729275, -0.03245744, -0.023625653, -0.010909856, 0.013556919, 0.014979406, 0.008225685, -0.0152144255, 0.004984889, -0.035549805, -0.002965576, 0.028449738, -0.015536031, 0.045272194, 0.00046076206, 0.0072917915, 0.045000065, -0.02862291, 0.0024213202, 0.011528329, 0.009635802, 0.015659725, -0.03225953, -0.026470626, 0.038765863, -0.02953825, 0.006895969, 0.03559928, -0.0140269585, -0.0032593505, 0.021522846, 0.0071000652, -0.009784237, 0.0034634464, -0.0043447698, 0.005649747, -0.03023094, 0.0023749347, 0.047696605, -0.04213035, 0.0071062497, 0.021324934, -0.04829034, 0.029216645, -0.0033397519, -0.035525065, -0.023155613, 0.03067624, 0.013111618, -0.015004145, -0.026050065, 0.019283975, 0.009147209, 0.01511547, -0.0396812, 0.031913184, -0.018232571, -0.0022713405, -0.0091966875, -0.049007766, -0.03520346, -0.016117396, -0.0063764523, -0.018071769, -0.037627872, -0.0068650455, 0.0464844, -0.0066423956, 0.025295528, 0.007570104, -0.009839899, -0.020211684, -0.014496997, 0.020100359, -0.025555287, -0.018034661, -0.0069207083, 0.008392673, 0.0033583061, 0.01963032, 0.042946737, 0.02230212, -0.022722682, 0.012054031, 0.023180353, -0.007180467, 0.018034661, -0.02503577, -0.0101243965, -0.010000702, -0.019259237, -0.020224053, -0.0058074575, 0.0014387218, -0.021794973, 0.041190274, -0.010625359, -0.011924151, -0.020706462, -0.042278785, 0.031715274, -0.0059125978, -0.0030166, -0.01756462, -0.014212499, -0.016624544, -0.011639654, -0.01262921, -0.041388184, -0.026693277, -0.043243602, -0.014781495, 0.0013583204, 0.046781264, 0.04487637, 0.021708388, 0.012901338, -0.0025125449, -0.09257297, 0.033323303, -0.007180467, -0.0058662123, -0.0040942887, -0.018677872, -0.009035884, -0.06659713, 0.0125055155, 0.0410171, -0.020854896, -9.742151e-06, -0.037998956, -0.03359543, -0.008739018, -0.037801042, 0.028128132, -0.03250692, 0.028993994, 0.0009609518, -0.026223237, -0.013940372, -0.0072732377, -0.040250197, -0.036588836, -0.023712238, 0.0014031597, 0.053633943, -0.0059435214, -0.059571277, -0.014175392, 0.014051697, -0.028598173, -0.0019342729, -0.0293156, 0.008541106, -0.0140269585, 0.018616024, 0.008473074, -0.032605875, -0.03631671, 0.01397748, 0.028227089, 0.020793049, -0.014163022, 0.012128247, -0.012610656, 0.0058260118, -0.011917966, -0.016575065, -0.021089915, -0.0055013136, -0.016711129, -0.021312565, 0.015090731, -0.037380483, -0.04641018, -0.019642688, -0.03829582, 0.003383045, 0.023130875, 0.01218391, -0.034881853, -0.04148714, 0.010223351, 0.022190796, -0.009308012, -0.029736161, -0.013198205, -0.031715274, 0.0077494616, 0.03995333, -0.013705352, 0.01781201, 0.05368342, 0.017007995, -0.011033551, 0.0116520235, -0.009277089, -0.05976919, -0.01116343, 0.009338936, -0.009524478, -0.027856005, 0.021609431, 0.026272716, -0.0031325636, -0.025357375, -0.023279307, 0.029365078, -0.017230647, 0.026297454, -0.035995103, 0.007922634, -0.020892004, -0.039112207, -0.0071557276, 0.004276738, -0.0023811194, 0.016265828, -0.0075020725, -0.011565438, 0.00036373918, 0.0030815396, 0.027460182, -0.008918375, 0.017737793, 0.02525842, -0.008027774, -0.0053374185, 0.0022759791, 0.0033552137, -0.00063857296, -0.008367934, -0.014484628, -0.017923335, -0.019073695, 0.01621635, -0.015498923, -0.023353525, -0.007897895, 0.009190503, 0.021386782, -0.011670577, -0.018851044, 0.015647355, -0.0012933807, -0.08772415, -0.024887336, 0.011243831, -0.0023362802, 0.024726534, -0.033941776, 0.019345822, -0.0016157596, -0.0146578, -0.033917036, 0.024454406, -0.016439002, 0.011021181, -0.005359065, -0.0071062497, -0.041635573, -0.00461999, -0.00642593, -0.016550327, -0.031987403, -0.043886814, 0.008745202, -0.017774902, -0.02909295, -0.06026397, 0.0023501958, 0.011917966, 0.0035036472, 0.0055229603, -0.00620328, -0.018034661, 0.02524605, 0.02956299, 0.0070938803, -0.019890077, 0.013940372, -0.00020854123, -0.003800514, 0.0014866535, -0.015461815, 0.052199084, -0.021127023, 0.050121017, -0.023613283, 0.026025327, -0.04945307, -0.0069207083, -0.031418405, -0.0008882812, -0.0045643277, 0.00202859, 0.04346625, 0.032309007, -0.016377155, 0.038172126, 0.033348043, 0.013804308, -0.01736671, 0.0030846319, 0.029835118, -0.011181984, -0.0024491514, 0.033273824, -0.0025712997, 0.007953557, -0.0077432767, 0.014484628, 0.026346931, -0.037627872, -0.029835118, -0.013544549, 0.0017765624, 0.00923998, 0.010916041, -0.02053329, 0.008788495, 0.010668652, 0.022252643, -0.042278785, 0.009233796, 0.022846377, 0.007916449, -0.010452187, 0.019716905, 0.053831853, 0.037281528, -0.021003328, -0.018653132, 0.012227203, 0.043441515, 0.06565705, 0.008640062, -0.013123988, -0.011887043, -0.043664165, -0.012598286, 0.025604764, 0.03067624, 0.026940666, -0.029142428, 0.049279895, 0.050022062, -0.033694386, -0.027089098, -0.02165891, 0.02095385, 0.012190094, 0.01713169, -0.020780679, -0.0056002694, -0.0002446832, 0.010476925, 0.017304862, 0.038741123, -0.006407376, 0.015189687, 0.052001175, 0.020892004, 0.019778753, -0.03112154, -0.003964409, 0.019382931, -0.009808975, 0.031888448, -0.05071475, 0.013903264, -0.018084139, 0.031517364, 0.025580026, 0.0054147276, -0.0009872369, -0.014422781, 0.021151762, -0.005479667, -0.032358486, 0.013779569, 0.009048254, -0.005442559, -0.045519583, 0.024330711, -0.0110644745, 0.009518293, -0.00857203, -0.009202872, -0.04618753, -0.007192836, -0.0053621572, -0.016290568, -0.009679096, 0.015053622, 0.040077023, 0.020879634, -0.014991775, 0.033966515, 0.0002659432, 0.020632245, -0.018368635, -0.007032033, -0.014051697, 0.021089915, 0.052495953, 0.04351573, 0.01804703, -0.01783675, 0.023699868, -0.03787526, -0.016166873, 0.029934073, -0.0032840895, 0.016525587, -0.012146802, 0.020001404, 0.009128655, 0.03673727, 0.01622872, 0.01646374, 0.00042056135, 0.043045692, -0.00011374097, 0.040324412, 0.021510476, 0.046360705, -0.012573548, -0.0034139687, 0.024998661, -0.041685052, -0.015140208, 0.03000829, -0.028573433, 0.017057473, -0.0057796263, 0.0024305973, 0.009561586, 0.047894515, -0.018034661, -0.013816677, 0.04596488, 0.008176208, 0.001662145, 0.0045086653, -0.0058229193, 0.019444779, 0.012913708, -0.007242314, -0.021609431, 0.020471442, -0.009468815, -0.0063702674, 0.04220457, 0.010446002, -0.0077680154, -0.014744386, -0.014967036, 0.02388541, -0.029835118, -0.010674837, -0.0037479438, -0.010446002, -0.025728459, 0.018405745, -0.0419077, -0.026544843, 0.01736671, -0.029191906, 0.015239164, -0.008479259, 0.012913708, 0.020805417, -0.010742868, -0.003614972, -0.02659432, -0.00012900951, 0.008887451, 0.0032964589, 0.023947258, 0.030305156, 0.012388006, -0.00018824759, 0.024231756, 0.026495365, 0.019370561, 0.016105026, -0.003707743, -0.016748236, 0.012196279, -0.005702317, 0.011924151, 0.023539066, -0.0056002694, -0.034881853, 0.017663578, 0.042501435, 0.020496182, -0.0124189295, 0.0232422, 0.023588544, 0.016179243, 0.03975542, 0.024540992, 0.00074525946, -0.006265127, -0.011528329, 0.0045364965, -0.028845562, 0.019048955, -0.011819011, -0.013804308, -0.00315421, 0.03653936, 0.009221426, -0.056849997, 0.032358486, 0.003766498, 0.036193017, 0.017243015, -0.049081985, -0.0073598237, -0.011373711, 0.003383045, -0.006561994, -0.028969256, 0.003587141, -0.014843342, -0.016154503, 0.0013474971, 0.010520219, -0.0065743634, -0.012975555, 0.003989148, 0.005158061, 0.023539066, 0.014843342, -0.0039273007, 0.0025682074, -0.031022584, 0.01217154, 0.011645839, 0.010804716, -0.01852944, -0.024207016, 0.02029827, -0.01160873, 0.0017255385, -0.018319158, 0.0020656984, 0.022277383, 0.013173466, -0.031962663, -0.01624109, 0.0051858923, 0.00093930523, 0.010019256, -0.006475408, 0.03785052, 0.031220496, -0.014732017, -0.0021924854, -0.024429668, 0.00925235, 0.0048271785, -0.0027552953, 0.0019110802, -0.02436782, 0.01757699, 0.00422726, -0.019914817, 0.039854374, -0.009598695, 0.006231111, 0.030948367, 0.0046416367, 0.023650391, 0.011262385, 0.004561235, -0.05165483, -0.02956299, -0.005368342, -0.019098433, 0.0017239923, -0.050393146, -0.013940372, 0.0019296345, -0.018467592, -0.0101305805, -0.0075391806, 0.0042829225, -0.021374412, 0.026470626, 0.017181167, -0.020063251, 0.051456917, -0.027658094, -0.038196865, -0.021572324, 0.014682539, 0.0017069844, -0.01621635, -0.013965111, -0.05054158, -0.0041128425, 0.005464205, 0.01556077, -0.005009628, -0.016674021, 0.028177612, -0.013185835, 0.00034402538, -0.0014936113, 0.036193017, 0.0037262973, 0.007192836, 0.019370561, -0.026668537, 0.0053405105, -0.016080286, 0.023650391, -0.012975555, -0.024491513, 0.024417298, 0.0043169386, -0.029637206, -0.02456573, -0.031517364, 0.032210052, -0.02007562, 0.011856119, -0.011367526, 0.014422781, -0.03592089, 0.02684171, 3.592456e-05, -0.01285186, -0.02029827, -0.019989034, -0.0010235722, 0.02684171, 0.012567363, -0.026000587, -0.005529145, -0.0025233682, 0.031146279, 0.0075886585, -0.00063779985, -0.015251534, 0.051951695, -0.03995333, -0.005795088, 0.012666319, 0.02795496, 0.015993701, -0.038493734, 0.008813234, -0.032284267, 0.017106952, -0.0058136424, -0.030577283, -0.003562402, 0.007483518, 0.027930222, 0.021028068, -0.018900521, -0.0017873858, -0.025382115, -0.005974445, 0.01916028, -0.004313846, -0.0067166123, 0.0105511425, 0.0010313031, 0.03129471, -0.012691057, -0.002625416, -0.03584667, 0.00031832012, 0.016735869, 0.00020622196, 0.024837859, -0.011274755, -0.020768309, 0.001415529, -0.0016327676, -0.00093466666, 0.017428556, 0.012901338, 0.011293309, -0.008967852, 0.0024862597, 0.0100439945, -0.003614972, 0.014299086, -0.007186651, -0.0024259586, -0.016129766, 0.0035314783, -0.001815217, 0.0045643277, -0.056008875, 0.0047869775, 0.02097859, 0.0089431135, 0.016698759, 0.0036458957, 0.03807317, -0.011219093, 0.003050616, 0.024540992, -0.008621507, 0.042056136, -0.0025419223, -0.07337559, 0.010328492, 0.011534514, 0.015251534, -0.014917559, -0.0007730907, 0.0015539123, 0.0052601094, 0.008677171, -0.022487663, -0.018875783, -0.0010498572, 0.013643505, 0.00073057075, -0.009295642, 0.013544549, -0.020162206, 0.0075020725, -0.0028650742, -0.01601844, 0.009258535, 0.01985297, -0.00032141246, -0.0069640013, 0.025357375, 0.0007719311, -8.639289e-05, 0.016970888, -0.002718187, -0.0064815925, 0.011318048, 0.0017224462, -0.015511292, 0.007279422, 0.0004456868, -0.028449738, -0.033348043, -0.0067537203, -0.03537663, 0.0062836814, 0.011955075, 0.034362335, 0.0032748124, -0.020681722, 0.012703427, 0.0068217525, 0.0072917915, -0.018826306, 0.023971997, 0.0279797, -0.0016806992, -0.016562697, -0.016760606, 0.00023018777, 0.027336488, 0.0062744045, -0.011645839, 0.015894745, 0.0035036472, -0.031072062, 0.015263903, 0.02050855, 0.020706462, -0.003519109, -0.00444991, -0.006228019, 0.010532588, -0.018566547, -0.005946614, -0.0026486088, 0.0016745145, 0.008955483, -0.032828525, -0.012777643, -0.015424706, 0.003001138, 0.025382115, -0.0018693333, 0.0059682606, 0.022834007, 0.0055105905, -0.012115878, 0.0042117983, 0.0009934215, -0.012678687, -0.028993994, 0.033892296, -0.01285186, -0.03359543, 0.014818603, 0.0034727235, -0.008374118, -0.018974738, -0.009617249, -0.0021878467, 0.049329374, -0.0396812, 0.01217154, -0.035995103, 0.012400375, -0.026520103, 0.010693391, 0.011423188, 0.01919739, 0.018220203, -0.03859269, 0.0021352766, 0.0030243308, -0.04537115, 0.026495365, 0.02325457, 0.042798303, -0.026742755, 0.008875081, 0.026891187, 0.011305679, -0.013507441, -0.009654357, -0.019135542, 0.020731201, 0.003757221, -0.04598962, 0.018665502, -0.0035593098, 0.04641018, 0.007916449, 0.03812265, -0.011039736, 0.023947258, -0.03765261, -0.0033954144, 0.037553653, 0.0043942477, -0.0077865697, -0.0245781, 0.008627692, -0.029365078, -0.027039621, -0.019048955, -0.0039489474, -0.014694909, -0.026099542, -0.023427742, 0.0028063194, -0.016426632, 0.015968962, 0.035104502, 0.011874674, -0.041215014, -0.03275431, -0.006364083, -0.0061321557, -0.019518994, -0.013829047, -0.0016884301, 0.029117689, 0.033100653, 0.020570397, -0.0202859, 0.019308714, -0.02592637, -0.017341971, -0.0055940845, 0.0042736456, 0.0004831817, -0.0011951983, -0.011849934, 0.011744794, 0.0024104968, 0.0101243965, -0.0035036472, -0.006518701, 0.019704536, 0.0396812, -0.04937885, 0.0012052484, -0.004357139, -0.0026795324, -0.038196865, -0.019778753, 0.044406332, 0.0074958876, -0.006357898, -0.008324641, 0.008287532, 0.018690242, 0.029587729, 0.0008001489, 0.008287532, -0.013655875, 0.009555401, 0.0027042713, -0.009641987, -0.024404928, -0.0008882812, -0.036440406, 0.0066423956, 0.024293603, -0.0018523254, -0.009425522, 0.040299673, -0.02211658, -0.0070691416, 0.017106952, -0.0049817967, 0.024491513, -0.024738902, 0.0097904205, 0.01825731, 0.014212499, 2.1296233e-05, 0.016760606, 0.02548107, 0.009641987, 0.0052879406, 0.010520219, -0.001832225, 0.008813234, -0.0061043245, 0.04504954, -0.015573139, -0.028152872, -0.021102285, -0.01622872, -0.014261978, -0.028969256, 0.014274347, -0.0055786227, -0.040077023, -0.017181167, -0.0119117815, 0.02277216, 0.027930222, -0.010433632, -0.0343376, -0.0092832735, 0.007706168, 0.016735869, -0.001458049, -0.033323303, -0.0365641, -0.02188156, -0.016810084, 0.06001658, -0.004848825, 0.00980279, -0.016006071, -0.03159158, -0.07144595, 0.008670986, 0.0010807809, -0.005751795, 0.037553653, 0.010105842, 0.0025712997, 0.011033551, -0.0061630793, -0.027460182, -0.040472846, -0.0057579796, 0.0041715973, 0.017651208, -0.03562402, -0.017069843, -0.004647821, 0.04937885, -0.03000829, -0.011188169, 0.030131985, 0.01960558], "content_hash": "6ba40353610070d5ee4aa3edebdef5ec"}, {"content": "Good for factual, predictable text. - **High temperature (e.g., 1.5)**: Spreads probability more evenly, making less likely tokens more probable. Good for creative, varied text. - **Temperature = 1.0**: The default, unmodified distribution. ## Greedy Decoding vs. Sampling Once we have probabilities, how do we pick the actual next token?\n\n- **Greedy decoding**: Always pick the token with the highest probability. Simple but can be repetitive.\n- **Sampling**: Randomly pick a token weighted by the probabilities. More varied but less predictable.\n- **Beam search**: Explore multiple possible sequences simultaneously and pick the best overall path. This is available in the dashboard's generation controls.\n\n## What You See in the Dashboard\n\nIn **Stage 5 (Output Selection)** of the pipeline:\n\n- The **predicted token** is highlighted after your prompt text, along with its confidence percentage\n- A **top-5 bar chart** shows the five most likely next tokens and their probabilities\n- A note explains how beam search and other techniques can influence the final selection beyond just the top-1 token", "source_file": "output_and_prediction.md", "chunk_index": 1, "embedding": [0.024105849, -0.007743178, 0.009488625, -0.008533298, 0.006805808, -0.016836742, -0.0059151268, 0.016290842, 0.00787247, 0.040511616, 0.022942217, 0.006665741, -0.052808758, -0.0077862754, 0.035569772, -0.022870388, -0.015112844, 0.010645074, -0.038184352, 0.03355856, -0.01818713, -0.029378105, 0.018172763, 0.031949587, 0.019523151, -0.018172763, 0.009014552, 0.05645768, 0.018273326, -0.01263474, -0.02588721, -0.042781416, -0.013906115, -0.04464897, -0.024824139, -0.004471362, -0.0181153, 0.024781043, 0.005645768, -0.03393207, -0.0060767424, -0.012814312, -0.04108625, 0.011758424, -0.010515781, 0.030943979, 0.007894019, -0.06625517, -0.007836555, 0.06556561, -0.03252422, 0.05372817, -0.029406836, 0.015012283, 0.020169614, -0.019753003, -0.056256555, -0.004873605, -0.016750548, -0.07470227, 0.017871082, 0.0036848332, 0.041057516, 0.04597063, 0.019106543, 0.0007762033, -0.051688224, 0.025844112, 0.001905267, 0.02004032, 0.052550174, 0.008461469, 0.016922938, -0.023488117, -0.03884518, -0.025599893, -0.012886141, -0.033299975, -0.03375968, -0.03987952, 0.014825528, -0.023057144, 0.011729693, -0.056199092, -0.013877383, -0.05002179, 0.02376107, -0.020069052, -0.038241815, -0.055825584, -0.058468893, -0.012972336, -0.022109, -0.0018442123, -0.008993004, -0.023128973, 0.026174527, -0.02526948, 0.00059573265, 0.0036650803, -0.004819733, -0.040482886, -0.03427685, -0.031748466, 0.07906948, -0.010968304, -0.060278986, 0.014602858, -0.04752214, -0.04108625, -0.07797768, -0.039908253, -0.043413512, 0.07274852, 0.018474447, -0.017440107, 0.023358826, 0.035253726, 0.039448544, -0.008511749, -0.039046302, -0.0351388, 0.005261482, 0.021606196, -0.029234447, 0.023502484, 0.024163311, -0.05789426, 0.0020148065, 0.018029105, 0.0072619226, -0.0015775467, 0.021979706, -0.017583765, -0.024996528, 0.0041589052, 0.011162243, 0.012146302, -0.016649988, 0.06361186, 0.014868625, -0.028386863, 0.008418371, -0.0038213085, -0.043413512, -0.031087637, -0.040310495, 0.02203717, -0.06211781, 0.03496641, 0.009445528, -0.027840963, -0.061543178, -0.0055092927, 0.025930308, -0.044304192, -0.02299968, 0.032093246, -0.023646142, -0.032236904, -0.02255434, 0.019178372, -0.0016457844, 0.006008505, -0.032495487, 0.011815888, -0.015313965, 0.029967103, -0.038557865, 0.016793646, 0.0151846735, -0.01540016, -0.0058935783, 0.00040448766, 0.01275685, 0.006191669, -0.057405822, -0.047780722, -0.0049023367, -0.032121975, -0.025197651, 0.031030174, -0.005300988, -0.0277404, 0.009093565, -0.02107466, 0.008562029, 0.00843992, 0.0040511615, 0.0040439786, -0.015888598, 0.00016498248, 0.051487103, -0.0135182375, -0.056227826, -0.017540667, 0.09659579, 0.034736555, 0.0059115356, -0.0025247931, 0.009639466, 0.0027061617, 0.020571856, 0.001292026, -0.010199733, 0.0052004275, -0.014179066, 0.007405581, -0.048642673, -0.03321378, 0.012361789, -0.030398078, 0.02781223, 0.015270868, -0.07159925, 0.012060107, -0.012972336, 0.072001494, 0.03255295, 0.0629223, -0.0078221895, 0.0027528505, 0.013094447, -0.0031407278, -0.02815701, 0.03215071, 0.0396784, -0.0058863955, 0.025585528, -0.0030275967, 0.038586598, -0.004636569, -0.0012489286, -0.01732518, -0.02014088, -0.008777517, -0.009316235, -0.015902964, 0.007042844, -0.013396128, -0.034851484, 0.009811856, 0.053354662, 0.016937304, 0.01633394, 0.010551696, -0.05981928, -0.025944673, 0.056428947, 0.06832384, 0.023588678, -0.048039306, -0.031547345, -0.02929191, 0.009107931, -0.030283151, -0.032639146, 0.0062060347, -0.011787157, 0.0029360147, -0.01677928, -0.0116004, -0.05821031, 0.0061306143, -0.025312576, -0.014710601, -0.021203952, -0.044620242, -0.0323231, -0.04651653, -0.036460456, -0.014983552, -0.013173458, 0.04390195, -0.054417733, -0.03416192, -0.013266836, -2.6571159e-05, -0.049533352, 0.02746745, -0.03881645, -0.015859867, 0.029004592, 0.008152603, -0.012699386, -0.009862136, 0.024651749, 0.0064430707, 0.020083418, -0.039793324, -0.005638585, -0.009072016, -0.009933965, -0.019408224, 0.020298906, 0.027840963, 0.017856715, 0.00070796564, -0.043269854, -0.022583071, 0.028300667, -0.060393915, -0.011363365, 0.018373886, 0.013712176, 0.016348304, -0.011621949, 0.02973725, -0.012570093, -0.012893325, 0.0026846128, 0.031576075, 0.0055200667, -0.016951669, -0.009423979, -0.03008203, 0.09027482, -0.050395302, 0.030024566, -0.061773032, -0.0036345527, -0.014459199, 0.04637287, 0.019997224, -0.007800641, -0.02114649, 0.021491269, 0.013453592, 0.00854048, -0.032236904, 0.016966036, 0.007064393, 0.014107237, 0.053756904, -0.046746384, -0.040367957, 0.038960107, 0.0076282513, -0.02578665, 0.041718345, -0.028544888, -0.02475231, -0.020241443, -0.035426114, -0.011140695, -0.025700454, -0.02860235, 0.04780945, 0.034535434, 0.024910334, -0.020816075, 0.014653138, -0.012591642, -0.009438344, 0.028746009, 0.006773485, 0.015170308, -0.01093239, -0.009933965, -0.0068920027, -0.02534131, 0.042120587, 0.022094633, 0.052808758, 0.026145795, -0.016132817, -0.05071135, -0.041229907, 0.0053189453, -0.009955514, -0.023286996, 0.018919786, -0.016204646, 0.0059977304, -0.052205395, -0.023990922, -0.0035824766, 0.0069674235, 0.0014150334, 0.05895733, -0.0101207215, -0.005530841, -0.02248251, -0.039247423, 0.030225689, 0.05634275, -0.0039685583, -0.0027223232, -0.02988091, -0.010178184, -0.007793458, -0.016132817, -0.008942724, 0.005742737, -0.014258077, 0.05459012, -0.046430334, 0.054647584, 0.0475796, 0.04042542, 0.030915247, -0.0033382578, -0.053067345, -0.0260596, -0.0063137785, 0.023258265, 0.036000747, 0.017727423, 0.05734836, -0.0034477971, 0.051343445, 0.016405769, 0.03939108, -0.06545068, 0.00705721, 0.032782804, 0.030943979, 0.045051217, -0.008461469, 0.038414206, -0.031576075, -0.045712043, -0.031001443, -0.037695915, -0.015084112, -0.00074612483, 0.016003525, -0.028171375, 0.017957278, -0.031374954, 0.024350068, 0.061428253, 0.046947505, -0.02709394, 0.031030174, -0.009079199, 0.0402243, 0.008842163, 0.018172763, -0.043614633, -0.010860561, -0.008418371, -0.050653886, -0.005663725, -0.06286483, 0.007987397, -0.04878633, -0.055337142, -0.012641923, -0.020557491, -0.0138558345, 0.022856021, -0.042781416, 0.047838185, -0.03025442, -0.039103765, 0.084011324, -0.0088134315, 0.036948893, -0.005103458, 0.021807317, 0.0035591323, -0.01275685, -0.02080171, -0.018100934, 0.008755968, 0.040511616, -0.00083142193, 0.03829928, 0.011923632, -0.007215234, -0.011176609, 0.019164005, -0.026706062, 0.0018962885, 0.0007268208, 0.05407295, -0.014983552, -0.0048556477, -0.010716903, 0.021476902, 0.026418746, -0.010666623, 0.043672096, -0.0011295127, 0.091194235, 0.025642991, -0.023919092, 0.018546276, -0.037638452, -0.009036101, -0.04597063, 0.02141944, -0.018129667, 0.008676956, -0.014049773, -0.0134966895, -0.0014285013, 0.0088134315, 0.0006415237, 0.039046302, -0.02867418, -0.038730256, 0.04878633, 0.037753377, 0.029090788, 0.051056128, 0.029938372, -0.0075923367, 0.03080032, -0.033673484, -0.063899174, -0.020787343, -0.0019088585, -0.031030174, -0.007215234, -0.0007344526, 0.024694847, 0.0054949266, 0.011212524, -0.0036112084, 0.01275685, 0.0077359946, 0.016693084, 0.004094259, -0.04375829, -0.029062057, -0.048240427, -0.0044138986, -0.00979749, 0.040655274, -0.03852913, 0.010235648, 0.04056908, -0.05387183, -0.023301363, -0.0038536317, -0.023315728, -0.022468144, -0.00015027996, -0.015270868, 0.021045929, -0.0060767424, 0.019365128, 6.543183e-05, -0.012570093, 0.00611984, 0.0001687984, -0.016693084, -0.009811856, 0.0075205076, -0.017526302, 0.018000374, -0.023416288, -0.016420133, 0.0034262484, 0.0045647398, -0.003395721, -0.002354199, -0.02815701, 0.021850415, -0.015874233, -0.0029575634, 0.0040511615, -0.0019878705, 0.0339608, 0.023660507, -0.0014617223, 0.027611108, -0.0073624835, -0.02403402, -0.0040188385, 0.0039470093, 0.014839893, -0.004765861, -0.027036475, 0.027682938, -0.0076282513, -0.004622203, 0.048441548, -0.011305901, -0.022439413, 0.010458318, 0.018833593, -0.013776823, 0.009043285, 0.017612496, 0.011162243, -0.017310815, -0.017483205, 0.044131804, -0.005484152, -0.022640534, 0.044189267, -0.0362306, 0.023746703, -0.018603738, -0.04197693, -0.03878772, 0.0064682113, 0.01313036, -0.03657538, -0.04752214, -0.0068417224, -0.012850227, 0.02598777, -0.045712043, 0.020772977, -0.0075061414, -0.00826753, -0.038730256, -0.04720609, -0.053986758, -0.027323792, 0.015773673, -0.01201701, -0.015199039, -0.0017247965, 0.0022446597, -0.006270681, -0.00035981892, 0.00427024, 0.003013231, -0.031058906, -0.022985315, -0.0022590254, 0.011758424, -0.010953939, -0.016247744, -0.020241443, -0.006223992, 0.008360908, 0.0055164755, 0.0071038986, -0.012512631, 0.02746745, 0.026461843, 0.054762512, -0.003474733, -0.030570468, 0.01966681, 0.007240374, -0.029995834, -0.008095141, 0.014322724, -0.0018163786, 0.011485474, 0.00080134347, 0.0064179306, -0.017971642, -0.030771589, -0.023057144, 0.05088374, -0.01846008, 0.0025858479, -0.0447639, -0.01708096, -0.019796101, -0.003943418, -0.012361789, -0.001424012, 0.002821088, -0.033644754, -0.033271242, -0.010573245, 0.05700358, 0.03200705, 0.020643685, 0.010853378, 0.019436955, -0.07694334, 0.033673484, -0.0060192794, -0.0115142055, -0.024881603, 0.0033741724, 0.014839893, -0.051199786, -0.008008946, 0.010156636, -0.023358826, 0.003731522, -0.055164754, -0.037638452, 0.021663658, -0.018072203, 0.045597117, -0.016736181, 0.01801474, 0.01110478, -0.0022859613, 0.00082468794, 0.010767183, 0.0140138585, -0.018057838, 0.0013800167, 0.005846889, 0.037552256, -0.009682564, -0.06941565, -0.009344967, 0.004295381, -0.0069422834, -0.034391776, -0.013388946, -0.01443765, -0.015486356, 0.0072583314, 0.011981095, 0.0010855173, -0.006349693, -0.01495482, -0.001338715, -0.00145005, -0.018905422, 0.016290842, -0.024651749, -0.004931068, 0.0034190654, -0.031403687, -0.012749666, 0.015428892, -0.043269854, -0.0046581174, -0.004079893, -0.03585709, -0.0015910147, -0.024393165, -0.04637287, 0.014926088, 0.01626211, 0.018431349, -0.012613191, -0.050797544, -0.019623712, 0.024321336, -0.028961496, -0.017684326, -0.00346755, -0.0035196263, 0.0073912153, 0.0027223232, -0.0004233428, 0.025700454, 0.007484593, -0.003111996, 0.004008064, 0.020514393, -0.018847957, -0.04651653, -0.012965154, 0.013532604, -0.016477598, -0.017957278, 0.04335605, 0.008597944, 0.014674687, -0.022511242, -0.012591642, 0.010989853, 0.009567637, 0.03146115, -0.021017198, 0.019278932, -0.030915247, -0.017770521, 0.020945368, 0.0004597063, 0.00787247, 0.0071793194, 0.0003079673, -0.0069279172, -0.0029773165, -0.006917143, 0.06924326, -0.0073912153, 0.015917331, 0.03217944, -0.03321378, -0.0035268092, -0.018144032, -0.018934153, -0.026030868, 0.003099426, -0.017296448, -0.0116004, -0.011011402, -0.0022500467, -0.0023919092, -0.048987452, -0.021534367, 0.0007155975, 0.025973406, 0.0035393792, -0.0226549, -0.002056108, 0.021778585, -0.12814313, -0.016563792, 0.023947824, 0.016175915, -0.010623525, -0.036000747, 0.04582697, -0.0052866223, -0.015098479, -0.028027717, -0.01455976, -0.016420133, 0.0017364686, 0.005085501, -0.018172763, -0.026318185, 0.030024566, -0.007534873, -0.004295381, -0.01715279, -0.049447156, 0.005609853, -0.036661576, -0.0116506815, -0.054877438, 0.0063065956, 0.0008888852, 0.008102323, 0.022238292, -0.008648224, 0.0015766489, 0.040253032, 0.02588721, 0.007994579, 0.01122689, -0.0034064953, -0.015558184, -0.022439413, -0.006166529, 0.0015155942, 0.036833964, -0.03131749, 0.043614633, -0.031604808, 0.020183979, -0.025025262, -0.02973725, -0.019164005, -0.009732844, 0.010063258, -0.02021271, 0.012412069, 1.1531939e-05, -0.0067806677, 0.01784235, 0.017124059, -0.00984777, 0.018618105, 0.0055236584, 0.019968491, -0.01381992, 0.003449593, 0.05372817, -0.017368278, 0.031777196, -0.025844112, 0.005347677, 0.017727423, -0.021678025, -0.017698692, -0.024378799, 0.0133386655, -0.009438344, 0.0055954875, -0.012376155, 0.015802404, 0.042034395, -0.0059259012, -0.031547345, 0.009488625, 0.021548731, -0.021836048, -0.036144406, 0.040138103, 0.030139493, 0.03131749, 0.009064834, -0.012577277, 0.0136978105, 0.0123474235, 0.028918399, -0.022669267, -0.014588492, -0.02403402, -0.050941203, -0.001087313, 0.021117758, 0.016319573, 0.04143103, -0.016118452, 0.047119893, 0.017267717, -0.017971642, -0.039448544, 0.001932203, 0.023646142, 0.0054626036, -0.0070536183, -0.024781043, -0.012088839, -0.00066486816, -0.0070212954, 0.009819039, 0.017397009, -0.031949587, 0.022496875, 0.047148626, 0.033098854, 0.013144727, -0.04235044, 0.015888598, 0.012972336, -0.0029827037, 0.010860561, -0.03358729, 0.005850481, -0.013719359, 0.013173458, 0.021462537, 0.0046724835, -0.0048843795, -0.00020302946, 0.0077647264, 0.0060767424, -0.0063461014, -0.0015928105, 0.017540667, 0.0061737117, -0.047895648, 0.021936608, -0.022353219, 0.016405769, 0.017871082, 0.009438344, -0.020816075, -0.027955888, -0.00890681, 0.0047119893, -0.01922147, 0.023085875, 0.022496875, 0.00826753, -0.0026935914, 0.019063445, -0.03442051, 0.03444924, 0.005085501, -0.016405769, -0.0028641857, 0.025312576, 0.03375968, 0.049993057, 0.034391776, -0.01633394, 0.034592897, -0.041862004, -0.013324299, 0.022396315, 0.0034406143, 0.014739333, -0.025671722, -0.023315728, 0.0037889853, 0.031547345, 0.014739333, 0.009474259, 0.03373095, 0.04947589, -0.006726796, 0.037408598, -0.007886836, 0.03444924, -0.01784235, 0.014854259, 0.030771589, -0.025456235, -0.0025876437, 0.026892819, -0.011557303, -0.01608972, -0.0062670894, 0.014329907, 0.008619493, 0.0453098, -0.036259335, -0.00041054824, 0.038960107, 0.013554152, -0.0006769893, 0.027611108, -0.039822057, 0.0028318625, 0.021232683, 0.009459893, 0.00114747, 0.034018263, -0.018747397, 0.0024475767, 0.040138103, 0.020298906, -0.039075036, -0.0006590321, -0.0070787584, -0.0021674433, -0.017799253, -0.013683445, -0.0003039269, 0.003903912, -0.015644379, 0.008806248, -0.019293299, -0.02939247, -0.0054123234, -0.008533298, 0.006647784, 0.0072044595, 0.020528758, 0.025183285, -0.02626072, -0.008461469, -0.023459386, -0.0118949, 0.036862697, 0.010329026, 0.01201701, 0.03126003, 0.0032161484, -0.019020347, 0.012498264, 0.006166529, 0.014825528, 0.011593218, -0.019968491, -0.016520694, -0.017454473, -0.032639146, 0.0021800133, -0.009567637, -0.0065149, -0.035081334, -0.00011038114, 0.028803471, 0.05895733, -0.0038823632, -0.0011690187, 0.005670908, 0.011643498, 0.037552256, 0.015055381, 0.0007959563, -0.0017993192, -0.0021243459, 0.008863712, -0.002618171, 0.02159183, -0.011937997, -0.019968491, 0.0018567824, 0.024120213, -0.011341816, -0.04444785, 0.034851484, 0.013877383, 0.040166836, -0.007800641, 3.083601e-05, -0.011241255, -0.027955888, -0.00022053781, -0.026016502, 0.008554847, -0.019178372, -0.0007667757, -0.012814312, -0.0096681975, -0.005534433, -0.028789107, -0.013604433, 0.030311882, -0.00055757345, 0.021649294, 0.008734419, -0.0023757478, -0.0016754139, -0.046459068, 0.0009750801, -0.027180133, 0.019954126, 0.010106355, -0.02152, 0.0047550867, -0.0008794576, 0.014825528, -0.00067474466, 0.011909266, 0.010745634, 0.033702217, -0.027826596, -0.010774366, -0.0025553205, 0.0260596, 0.0008691322, -0.01653506, 0.027553646, 0.016391402, -0.034937676, 0.009122296, -0.016017891, 0.016046623, -0.0096466495, -0.0006092006, -0.011607584, 0.0036058212, 0.028415594, -0.008547664, 0.00084129843, 0.029191349, 0.004180454, 0.00969693, 0.0351388, 0.011219706, 0.020126516, 0.023301363, 0.013410495, -0.022769827, -0.006830948, -0.027180133, 0.0007708161, 0.009215674, -0.0339608, 0.01137773, 0.004259466, -0.020198345, -0.0072547398, 0.004938251, 0.0011681209, 0.024824139, -0.0118518025, 0.017684326, -0.01677928, 0.074817196, -0.010530147, -0.043672096, -0.005530841, 0.0068740454, -0.016592523, -0.010329026, -0.017353913, -0.014049773, 0.0039649666, 0.0028534112, 0.010896475, 0.0109754875, -0.017827984, 0.021735488, -0.02623199, -0.0030186183, -0.0032071697, 0.038586598, -0.010084807, 0.026907183, 0.021893512, -0.0057858345, 0.012957971, -0.012419253, 0.014710601, -0.017698692, -0.014703418, 0.011586035, 0.024623018, -0.008454286, -0.028832203, -0.015845502, 0.0058576637, -0.026706062, -0.009172577, -0.023646142, 0.025197651, -0.020183979, 0.016060987, -0.01428681, -0.011779973, -0.002993478, -0.011887717, 0.023646142, 0.027050842, 0.023603044, -0.007193685, 0.002004032, -0.025973406, 0.015759306, 0.00708235, -0.017598132, -0.016003525, 0.037839573, -0.0447639, -0.0023829306, 0.022942217, 0.014021042, 0.0169804, -0.03778211, 0.008174152, -0.027955888, -0.006277864, 9.416347e-05, -0.039276157, 0.015658746, 0.008095141, 0.009208491, 0.0305992, -0.018402617, -0.0028965087, -0.02413458, 0.010048892, 0.03410446, 0.004212777, 0.009488625, -0.008698504, -0.0003375968, 0.020456929, -0.014882991, -0.018546276, -0.05125725, -0.014969186, -0.0071110814, 0.026504941, 0.04895872, -0.0067088385, -0.01856064, 0.009172577, -0.036747772, 0.0014626202, 0.028904032, 0.04967701, -0.001220197, -0.027251963, -0.00067968294, 0.0052076103, -0.0075779706, 0.0006374833, 0.0050926837, -0.001258805, -0.020155247, 0.011320267, -0.02860235, -0.018790495, -0.06907087, -0.01228996, 0.009201309, 0.00029360148, 0.005243525, 0.0114711085, 0.038500402, -0.007136222, -0.017095327, 0.028961496, 0.004191228, 0.02087354, -0.011988278, -0.025872845, 0.012555728, 0.03045554, 0.018603738, -0.022611802, 0.0006949466, 0.0021117758, 0.0032897731, 0.009991429, -0.0096251005, -0.021390708, 0.0021602605, 0.015586916, 0.015701843, 0.0041229906, -0.011837437, -0.015529453, -0.0021243459, 0.004162497, -0.011305901, 0.023674874, -0.008109506, -0.009438344, -0.030714126, -0.024852872, 0.008432738, 0.025585528, 0.008597944, -0.008360908, -0.0065795463, 0.015428892, -0.0049095196, -0.025183285, 0.014969186, 0.006033645, 0.003792577, -0.029248813, -0.030714126, -0.029794713, 0.013489506, 0.01364753, 0.016348304, 0.003560928, -0.03585709, 0.000646013, 0.024393165, 0.009941149, -0.02692155, -0.006985381, 0.0045755142, 0.002650494, -0.015773673, -0.032265633, -0.017454473, 0.027711669, 0.016750548, -0.0038751804, 0.0063532847, 0.0063137785, -0.0076067024, 0.0464016, -0.006744753, 0.010961122, 0.024192043, -0.004643752, -0.014739333, 0.02275546, -0.019451322, -0.010214099, -0.002354199, 0.013115995, 0.03045554, -0.01993976, -0.02516892, -0.011765608, 0.007621068, 0.016017891, -0.006324553, -0.012498264, 0.01308008, 0.0015972997, -0.02534131, -0.007692897, 0.014466383, 0.0049705743, -0.0113130845, 0.023315728, -0.010551696, -0.042264245, -0.00854048, 0.007251148, -0.015773673, -0.025398772, -0.0057463287, -0.016420133, 0.053670708, -0.042407904, 0.0039901068, -0.018661203, 0.010393672, -0.025499333, 0.020686783, 0.001015484, 0.009028919, 0.022080267, -0.02798462, 0.036259335, 0.020485662, -0.029306276, 0.022065902, -0.013590067, 0.019925395, -0.03252422, -0.0305992, 0.042953804, 0.016491963, -0.005394366, -0.037494794, -0.018043472, 0.010925207, -0.011880535, -0.049562085, 0.01149984, -0.0023775434, 0.04214932, -0.0051860614, 0.054647584, -0.015874233, 0.007064393, -0.017713057, 0.0005225568, 0.029248813, 0.0078221895, -0.028746009, -0.035081334, -0.004536008, -0.05019418, 0.0051285983, -0.0074199466, -0.02988091, -0.006141389, -0.012957971, -0.003929052, -0.015687477, -0.011140695, 0.020945368, -0.007908384, 0.02368924, -0.034018263, -0.031202564, 0.030829053, -0.013834286, 0.010336208, -0.005638585, -0.015788037, 0.03884518, 0.043068733, 0.022080267, -0.009890868, 0.009617917, 0.0029234446, -0.0034064953, -0.00076991826, -0.0044067157, -0.0016799032, 0.0118302535, -0.014459199, -0.0066549666, 0.024766676, 0.0022500467, -0.033673484, 0.00890681, 0.03217944, 0.012519813, -0.022468144, 0.013238104, 0.011025768, -0.007915568, -0.022741096, -0.012225314, 0.015270868, -0.024464995, -0.016664352, 0.0004713785, -0.0099196, 0.016649988, 0.011758424, 0.011032951, 0.014150334, 0.009661015, -0.01263474, 0.036431722, -0.010874927, -0.027065208, -0.0260596, -0.0043348866, 0.040655274, 0.019709907, -0.019365128, -0.013245287, 0.014035407, -0.042235516, 0.013051349, 0.012670655, -0.008547664, 0.006141389, -0.023588678, -0.009165394, 0.0109754875, 0.025729187, -0.019982858, 0.016836742, 0.034707826, 0.03166227, -0.025743552, 0.005631402, -0.015069746, 0.00021638519, 0.0057858345, 0.050596423, -0.023875995, -0.0075061414, -0.0138558345, -0.017138425, -0.039649665, -0.012146302, 0.059589427, -0.018790495, -0.020169614, 0.031001443, -0.0027743992, -0.003925461, 0.009682564, -0.0094670765, -0.041172445, 0.0057822433, -0.027424352, 0.031949587, 0.014768064, -0.0063065956, -0.026002137, 0.00251761, -0.015270868, 0.027165769, 0.0069135516, -0.0058935783, 0.0098262215, -0.014897357, -0.057808064, -0.006931509, 0.017253352, 0.0057355543, 0.021994073, 0.016305206, -0.020442564, 0.004510868, 0.005056769, -0.03884518, -0.018144032, -0.0113130845, -0.013051349, 0.0356847, -0.014660321, 0.009768759, 1.6175578e-05, 0.05002179, -0.044390388, -0.025757918, 0.0283294, -0.006238358], "content_hash": "f4d060fd4556e9d8b101910303cfe309"}, {"content": "# Pipeline Stages\n\n## Overview\n\nThe pipeline visualization shows the 5 stages your text passes through inside the transformer model. A flow indicator at the top shows the path: **Input \u2192 Tokens \u2192 Embed \u2192 Attention \u2192 MLP \u2192 Output**. Click any stage to expand it and see details.\n\n## Stage 1: Tokenization\n\n**Icon**: Puzzle piece | **Summary shows**: \"X tokens\"\n\nThis stage displays how your input text was split into tokens. Each row shows:\n- The **token** (the text piece, displayed in a blue box)\n- An arrow pointing to its **ID** (the number the model uses internally, in a purple box)\n\nNotice that spaces are often attached to the beginning of the following word (e.g., \" cat\" with a leading space). This is normal for models like GPT-2 that use BPE tokenization.\n\n## Stage 2: Embedding\n\n**Icon**: Cube | **Summary shows**: \"X-dim vectors\"\n\nThis stage shows how token IDs are converted into numerical vectors using a pre-learned embedding table. You'll see:\n- A visual flow: Token ID \u2192 Lookup Table \u2192 Vector\n- The embedding dimension (e.g., 768 for GPT-2)\n- An explanation of how the lookup table was learned during training\n\n## Stage 3: Attention\n\n**Icon**: Eye | **Summary shows**: \"X heads \u00d7 Y layers\"\n\nThis is the most detailed stage. It includes:\n- **Head Categories**: Attention heads are automatically categorized by their behavior pattern (Previous-Token, First/Positional, Bag-of-Words, Syntactic, Other). Click each category to see which specific heads belong to it.\n- **BertViz Visualization**: An interactive attention map showing which tokens attend to which. Lines connect tokens on the left to tokens on the right. Thicker lines mean stronger attention.\n\n**Navigating BertViz**: Single-click a head square to select/deselect it. Double-click to show only that head. Hover over tokens or lines to see exact attention weights.\n\n## Stage 4: MLP (Feed-Forward)\n\n**Icon**: Network | **Summary shows**: \"X layers\"", "source_file": "pipeline_stages.md", "chunk_index": 0, "embedding": [0.0019218383, 0.023723403, -0.0063646673, -0.027934154, -0.031914346, -0.04448593, 0.005399956, 0.04958252, -0.017376937, 0.07023583, 0.046184793, 0.00021444348, -0.014343254, -0.04671872, -0.0022570603, 0.0038618788, 0.0011975465, 0.039146647, -0.0034219946, 0.07421602, 0.035554767, -0.024827663, 0.0061219726, 0.047471073, 0.008372965, 0.024427217, -0.00019586217, 0.05315013, 0.009574304, 0.0063039935, -0.01743761, -0.018808836, -0.014549544, -0.0051056887, 0.012814278, -0.015580997, -0.02674495, 0.06475093, -0.030822221, 0.035967346, 0.011734286, 0.008542852, -0.054217987, 0.013081241, -0.00030374754, 0.04994656, 0.028662238, -0.031137723, -0.009962616, 0.05814964, -0.008761277, 0.047082763, -0.026623603, -0.014695161, 0.02502182, 0.0028016064, -0.007620612, -0.026235292, -0.01741334, -0.04676726, 0.017109973, 0.015738748, 0.01497426, 0.005679055, 0.011552266, 0.011831365, -0.077953525, 0.02133286, -0.011345975, 0.02066545, 0.062469605, 0.0146587575, -0.029875712, 0.02548294, 0.04960679, 0.0025725632, -0.034244217, 0.047398265, -0.016454697, -0.03800598, 0.03128334, -0.058926262, 0.008512515, -0.069362134, -0.007596343, -0.05179104, -0.029341783, -0.041403707, -0.045942098, -0.05766425, -0.011352043, 0.016903684, -0.034681067, -0.00085777394, 0.034195676, -0.012790008, 0.007839037, -0.0076630837, -0.017656036, 0.02338363, 0.06431408, -0.0075113997, -0.026550796, -0.00829409, 0.044583008, -0.021466343, 0.022946779, 0.038272947, -0.027060455, -0.03936507, -0.08795254, 0.0018611646, 0.026016867, 0.036525544, 0.043636497, -0.007535669, 0.026259562, 0.004435245, 0.01592077, -0.021624094, -0.042908415, -0.015629536, 0.026599335, 0.006225118, -0.014585949, 0.0038952492, 0.025288783, 0.00036006028, 0.0040044617, -0.032011423, -0.04533536, -0.01741334, 0.016066385, -0.029171897, -0.0065648905, -0.028832125, -0.0335404, 0.03504511, -0.017825922, 0.030264024, -0.0052816425, -0.021684768, 0.004589963, -0.013785057, -0.02720607, 0.031623114, -0.037957445, 0.061450288, -0.041063935, -0.017498285, 0.011066876, 0.014197637, -0.03351613, -0.04819916, -0.036598355, -0.019718941, -0.000510038, 0.0572274, 0.023007452, -0.031186262, 0.010247782, 0.0066012945, -0.0070624147, -0.0127171995, -0.059314575, 0.039632037, -0.014039886, -0.015908634, -0.040772703, -0.06790596, -0.022279369, -0.042884145, -0.013748652, -0.02584698, 0.021138703, -0.0148043735, -0.06945921, -0.062323987, 0.0052270363, -0.04686434, -0.014318985, 0.0020082982, -0.052033734, -0.02130859, -0.032957934, 0.005093554, -0.0035585104, 0.05392675, -0.010975866, 0.024609238, -0.0035736787, 0.045480978, 0.009016107, -0.0018808836, -0.02584698, -0.0020720055, 0.056547854, -0.01569021, 0.022218695, 0.018942317, 0.03577319, 0.042956952, 0.017583227, -0.019500516, 0.049533978, 0.017122108, -0.032933664, 0.008057462, -0.017898731, 0.0037192956, 0.017231321, -0.020070847, 0.022546334, 0.02815258, -0.045456707, -0.02176971, -0.06572171, 0.058343794, 0.034171406, 0.0126443915, 0.059702884, 0.008664199, 0.018808836, 0.011248897, -0.037350707, 0.028735047, 0.034025792, -0.035991617, 0.00679545, 0.004410975, -0.012419899, 0.042762797, -0.016915817, 0.017461881, 0.0030746378, -0.044340312, -0.0015373189, -0.024682047, 0.023517111, -0.023456438, -0.016114924, 0.043296725, 0.032084234, 0.022910375, 0.033783097, 0.02499755, -0.0023586887, -0.042956952, 0.039753385, 0.037957445, 0.037836097, 0.021247918, -0.04805354, 0.013263263, -0.0059611876, -0.013785057, -0.019427707, 0.022267234, 5.0814193e-05, 0.0012422933, 0.040578548, -0.004565693, -0.04353942, -0.010745306, -0.0021766676, -0.011139684, -0.027303148, -0.011801028, -0.012438101, -0.05407237, -0.0038558112, 0.009331609, 0.014379658, 0.04082124, -0.044534467, 0.0093255425, -0.0009449923, 0.03189008, -0.029171897, 0.031186262, -0.011740354, -0.01923355, 0.036865316, 0.050237793, -0.019961635, 0.0024588, 0.029244706, -0.018396255, 0.0055668084, -0.0083122915, 0.02384475, 0.012705065, 0.047859386, 0.018905913, 0.004520188, 0.018978722, 0.034705337, 0.026987646, 0.010211377, -0.0075599384, 0.045602325, -0.084748976, 0.030069867, 0.034025792, 0.008779479, 0.032957934, -0.0035767125, -0.044777162, 0.001953692, -0.021417804, -0.0045717605, -0.048320506, 0.030652335, -0.059848502, -0.0053392826, 0.009756325, 0.03946215, -0.008561054, 0.022085214, -0.023092397, 0.0008608076, -0.016296946, -0.019694671, -0.0026362706, -0.006789383, -0.0047841184, -0.016369754, 0.05708178, 0.017498285, -0.024669912, 0.036501277, -0.042156063, -0.028735047, -0.012850682, -0.05392675, -0.002000714, 0.006376802, 0.052955974, -0.022740489, 0.04895151, -0.07198323, -0.017983673, -0.041112475, -0.010417668, -0.05353844, 0.06441116, 0.0018990857, -0.006922865, 0.015144146, -0.0021023424, -0.02384475, 0.00910105, 0.013651574, -0.0141369635, 0.0027439664, 0.006082535, 0.00022525097, 0.00020458401, 0.008724873, -0.000875976, -0.021721171, -0.00863993, 0.014889317, 0.018942317, -0.021915328, -0.0121954065, -0.0045747943, 0.01681874, -0.024584968, -0.020580506, 0.021466343, 0.012705065, 0.039753385, -0.0026954275, -0.017498285, -0.0151805505, 0.0027834042, -0.011170021, -0.05140273, 0.0381516, -0.0027803706, 0.006370735, -0.022109482, 0.0034826682, -0.0012809727, 0.043442342, 0.030264024, 0.0013037253, -0.02502182, 0.0381516, 0.00060787424, -0.02628383, -0.004550525, 0.020908145, 0.012632256, 0.01006576, -0.024160253, 0.033394784, 0.0286137, 0.036525544, 0.01766817, -0.018736027, -0.061159052, -0.04130663, 0.0286137, 0.014513141, 0.03407433, -0.013299667, 0.01810502, 0.011018338, 0.04222887, -0.0056638867, -0.003561544, -0.028055502, -0.033297706, 0.02997279, 0.02584698, 0.014088425, -0.0035463755, 0.048417583, -0.016879413, -0.026526526, -0.03890395, -0.010484409, -0.03621004, 0.013069107, 0.062275447, -0.05048049, 0.0047628824, 0.00930734, 0.0148043735, 0.051888116, 0.027351687, 0.004471649, 0.0068803933, -0.054703373, 0.02856516, 0.03397725, -0.02545867, -0.011576535, 0.014391793, -0.048029274, -0.04807781, -0.05120857, -0.032957934, 0.012656527, -0.012171137, -0.0763032, 0.0058277054, -0.02538586, 0.010217445, 0.046937145, -0.0062008486, 0.044752896, -0.017522555, -0.043223917, 0.07353648, 0.018942317, 0.040942587, -0.027497305, 0.03310355, 0.024779124, 0.010053626, 0.0043047965, -0.013675843, -0.001236226, -0.013967077, -0.047835115, 0.036938127, 0.013493823, -0.015386841, -0.011619006, -0.010059694, -0.01661245, 0.012826413, -0.012019453, 0.014124829, -0.026550796, -0.031040646, -0.018905913, 0.016891548, 0.015860096, -0.032933664, -0.0074628606, -0.0029366054, 0.091787115, -0.05538292, -0.012972029, 0.013518092, -0.011837432, 0.0021614993, -0.02499755, 0.02246139, -0.012031588, -0.004671872, -0.044413123, -0.07756521, -0.046621643, -0.012802143, -0.023747671, 0.006971404, -0.030919299, 0.0035979482, 0.01933063, 0.043515153, -0.013979212, 0.044704355, 0.050286334, 0.0026848095, 0.037059475, -0.05548, -0.0125715835, 0.0146587575, -0.0032248052, -0.0031489632, 0.028128311, 0.024427217, -0.0003867946, 0.020859605, 0.009762392, 0.0126443915, -0.0014463085, 0.034753874, 0.0032612095, -0.008172742, -0.007123088, -0.050237793, -0.036913857, -0.029875712, -0.017449746, -0.00031607188, -0.015398975, 0.002528575, -0.017401207, -0.06407139, -0.0035221062, -0.03084649, -0.033758827, -0.04358796, 0.01635762, 0.003570645, 0.0070806164, 0.0017868393, 0.030021328, -0.002020433, 0.015095607, -0.0018050415, 0.016272677, 0.026186753, 0.024609238, 0.015047069, -0.029196167, 0.028977742, 0.0018293109, -0.0019006025, -0.006825787, -0.006625564, -0.0014197638, 0.009877672, -0.025628556, 0.018602544, 0.010308456, 0.019148609, -0.020216465, -0.017886596, 0.026890568, 0.040796973, -0.014464601, 0.011928442, -0.021199377, -0.022279369, -0.022534199, 0.02810404, 0.017692441, -0.012607987, -0.026526526, -0.0062372526, 0.039170917, -0.01635762, -0.0026681244, -0.0393408, -0.03713228, -0.017425476, -0.016576044, 0.010083963, -0.0005544056, -0.012159002, -0.0147315655, -0.00040689277, 0.004080304, 0.03307928, -0.02382048, -0.020362081, 0.013166185, -0.006043097, -0.028055502, 0.02633237, -0.011261032, -0.04715557, -0.012753604, -0.0010026323, -0.017886596, -0.021405669, 0.011182156, -0.031234803, 0.013105512, -0.023189474, -0.014173368, 0.0060279286, 0.01584796, -0.020580506, -0.046403218, -0.023953963, -0.013566631, -0.014488871, -0.005363552, -0.008591391, 0.0022707118, 0.0032278388, -0.02112657, -0.01294776, -0.0068318546, -0.0012802143, -0.03904957, 0.006255455, -3.867946e-05, 0.006643766, -0.017595362, -0.019718941, 0.03363748, -0.026575064, 0.023274418, 0.01886951, 0.0029912116, -0.0031186263, 0.0090646455, 0.00850038, 0.002384475, -0.008737007, -0.026138214, 0.021065896, -0.007657016, -0.019160744, -0.017862327, 0.0052513056, 0.0038952492, -0.009198127, 0.017704574, -0.034705337, -0.030555258, -0.02900201, -0.012899221, 0.035360612, -0.005418158, -0.00335222, -0.015083473, -0.04091832, -0.011430918, -0.009483294, 0.02676922, -0.023917558, 0.015496054, -0.034729604, -0.018796701, -0.02158769, 0.026405178, 0.024839798, 0.018893778, 0.022449255, 0.006049164, -0.041864827, 0.04630614, 0.016248407, 0.030215485, -0.0028971673, -0.02382048, 0.018444793, -0.038734067, 0.0027318317, 0.017983673, -0.018529737, 0.009107117, -0.036622625, -0.00070950267, -0.011036539, -0.01338461, 0.0034614324, -0.014610218, 0.016636718, -0.008476111, -0.030773683, -0.0050025433, 0.003007897, -0.022570603, 0.00081530237, 0.020968817, -0.0013575732, 0.054169446, -0.014403928, -0.06479947, 0.0001827794, -0.0158965, -0.058489412, -0.014246176, -0.02200027, -0.0054909666, -0.015423245, 0.03366175, 0.0115644, -0.029366054, -0.044825703, 0.0076388144, -0.009804864, -0.011000135, -0.024208792, 0.03043391, -0.020313542, -0.02771573, -0.0051481603, -0.0025725632, -0.015508188, 0.027448766, -0.030992107, 0.01078171, -0.0013295116, -0.04261718, -0.029560208, -0.018250639, 0.0014493421, 0.0023905423, 0.019342763, 0.023310822, -0.013785057, -0.066255644, 0.0012096812, 0.03264243, -0.018760297, -0.0003494424, -0.052422047, -0.012680796, -0.038806874, 0.00050548743, -0.020677585, 0.023784077, -0.021320725, -0.022607006, -0.01840839, 0.033297706, -0.00976846, -0.0231288, -0.011782825, 0.008700604, -0.02223083, -0.010884855, -0.01453741, 0.014707296, 0.022971049, -0.0062190504, 0.03504511, -0.0027652022, -0.0025740801, 0.023055993, -0.006850057, 0.0052422043, 0.011236763, -0.009829134, 0.03225412, 0.0056426506, 0.040020347, 0.010411601, 0.0033764893, -0.03587027, 0.016709527, -0.009295206, 0.049533978, -0.0066619683, 0.038782604, -0.02900201, -0.010678565, -0.0040044617, -0.0036737903, -0.0063039935, 0.007274772, -0.04552952, 0.01569021, -0.0067772483, -0.028953472, -0.004107607, -0.00817881, -0.011030472, -0.06266376, -0.019852422, 0.044752896, -0.054460682, -0.015192686, -0.01382146, 0.0065163514, -0.09207835, -0.011097213, 0.010842384, -0.021272186, -0.013129781, -0.020495564, 0.018396255, 0.0028880665, -0.017425476, -0.017583227, 0.026866298, 0.0011884454, -0.022121618, -0.00011793443, 0.019828154, -0.043054033, 0.05276182, 0.0017762214, 0.01340888, -0.03730217, -0.027618652, -0.006492082, 0.034171406, 0.005630516, -0.047082763, 0.021466343, -0.014743701, 0.0079785865, 0.028055502, 0.020531967, -0.027497305, 0.03186581, 0.006710507, 0.0067833154, 0.051111493, -0.0035676113, -0.015253359, 0.008542852, 0.030919299, 0.021818249, 0.014003482, -0.025337322, 0.019816019, -0.02412385, 0.040020347, 0.011479457, -0.029633017, -0.029171897, 0.024500025, 0.028880663, -0.035821732, -0.019112203, -0.009756325, -0.0170493, 0.0526162, 0.007687353, -0.017558958, -0.0196704, 0.03392871, 0.013712248, -0.0075235344, 0.036549814, 0.025361592, 0.024245195, 0.026429448, 0.003188401, 0.012341023, 0.025046088, -0.019682536, -0.02907482, 0.013979212, -0.010399466, 0.008864422, -0.026453717, -0.0015024316, -0.0020507697, 0.00413491, 0.009119252, -0.0019658266, -0.017012896, 0.037593402, 0.01520482, -0.02548294, 0.0052907434, 0.010721036, 0.017401207, -0.008482178, -0.047519613, 0.021260051, 0.008979702, 0.004225921, 0.0014766452, -0.0057124253, -0.012031588, -0.021891057, -0.009968683, 0.024766989, 0.03363748, 0.06635272, -0.0017534689, 0.062275447, 0.00647388, -0.033831634, -0.025749903, -0.04720411, 0.010223513, -0.0051390594, -0.0030321663, -0.036743972, -0.03630712, 0.018092886, -0.018736027, 0.03361321, 0.02589552, -0.027497305, 0.024038905, 0.03907384, 0.0075113997, 0.0032430072, -0.043005493, -0.005075352, 0.014634487, -0.03710801, 0.018359851, -0.028298197, -0.016442563, -0.014780104, 0.019658267, -0.014088425, -0.010241714, -0.019597594, 0.013505957, -0.018990858, -0.010903058, -0.01292349, 0.014634487, 0.012080126, 0.031477496, -0.013141915, 0.009701719, -0.004231988, -0.00061849214, 0.02858943, 0.0332249, -0.019197147, 0.030191215, -0.024196656, -0.010939461, 0.0007178453, 0.013857865, 0.02638091, 0.041525055, -0.0020659382, 0.01497426, -0.007001741, 0.022509929, -0.0036980598, -0.03397725, -0.035166454, 0.03489949, 0.02292251, 0.032399736, 0.021599824, -0.017316263, -0.0005434085, -0.011212493, -0.01269293, 0.02130859, 0.0047355792, 0.0067651137, -0.03315209, 0.022291504, 0.043903463, 0.023990367, 0.010466207, 0.019221416, 0.0016139194, 0.06135321, 0.015095607, 0.016952222, -0.011321706, 0.027084723, -0.0071352227, 0.003637386, 0.026526526, -0.003570645, 0.0019006025, 0.020689718, -0.022873972, -0.002355655, 0.0066194967, 0.0116736125, 0.005581977, 0.033297706, -0.016527506, -0.008069597, 0.004031765, -0.0074264565, -0.022364313, 0.037520595, 0.0011436986, 0.026065405, 0.031210532, -0.0024588, -0.025167435, 0.025240244, -0.009349812, -0.007620612, 0.004759849, 0.030967837, 0.014294715, 0.003437163, -0.0067772483, 0.02582271, -0.02276476, -0.0047477144, 0.04038439, -0.004356369, 0.02638091, 0.008670267, -0.030336833, -0.032084234, -0.021162974, 0.009404418, -0.0133118015, 0.0064799474, 0.0029153696, 0.007305109, -0.011303503, -0.0053787204, -0.008027126, 0.011534063, -0.014088425, 0.01840839, 0.01997377, 0.0023814412, 0.03128334, -0.020859605, 0.03502084, -0.014246176, 0.035239264, 0.030264024, -0.031040646, 0.021842519, 0.019427707, 0.011030472, 0.019706806, 0.016527506, -0.030021328, -0.013930673, -0.013882134, 0.039607767, 0.0068803933, -0.041209552, 0.022158021, 0.011782825, 0.038394295, 0.022048809, -0.022910375, 0.0056729876, 0.011655411, 0.03951069, 0.025361592, -0.032399736, 0.013190455, 0.0014531342, -0.020713989, 0.018092886, 0.026623603, 0.028856395, -0.0068075852, 0.026939107, -0.011879903, 0.002546777, 0.0033643546, 0.00658916, 0.006407139, -0.020531967, 0.024087444, -0.022291504, -0.0028167749, -0.028201118, 0.02905055, -0.026526526, 0.001685211, 0.028735047, -0.020653315, -0.0054424275, 0.011097213, -0.025774172, -0.018517602, 0.001838412, 0.009404418, -0.0003763663, -0.00817881, 0.017134242, -0.008203079, 0.0062493873, -0.008840153, -0.016988626, 0.008324427, -0.001962793, 0.0027940222, 0.0009131386, -0.0079785865, 0.0036434536, 0.020750392, -0.053441364, -0.017910866, 0.0012536696, -0.005985457, 0.0054090573, -0.00043533352, 0.04535963, -0.0060066925, 0.0064678127, 0.011825297, -0.03630712, 0.009289138, -0.03395298, -0.005933884, 0.024548564, -0.008882624, 0.025337322, -0.019985905, -0.00863993, 0.023286551, -0.009428687, 0.033443324, 0.003734464, 0.03189008, -0.005484899, 0.018189965, 0.016940087, -0.02905055, -0.025264513, -0.03138042, -0.0061614104, -0.01006576, -0.0245607, 0.03863699, 0.01766817, -0.016369754, -0.0041197417, 0.027545843, -0.02066545, 0.0142097715, -0.019549055, 0.022315774, 0.005005577, 0.051111493, -0.017474014, -0.0060886024, -0.0040074955, -0.0062190504, 0.0038163734, -0.00018116776, -0.0044595143, -0.023686998, -0.016466832, 0.001800491, 7.4941454e-05, 0.0035099715, -0.018347716, 0.024900472, -0.015301898, -0.013967077, 0.014501005, 0.043685038, 0.02200027, 0.0033431188, 0.014318985, -0.0130205685, 0.0028698642, -0.014840778, 0.012177205, 0.0015312516, -0.013348206, 0.012401697, -0.014428197, -0.014185502, -0.07067268, 0.0211023, 0.04135517, -0.022861836, 0.02018006, 0.0055031013, 0.022934645, -0.01430685, 0.0052604065, 0.012535179, -0.022558467, 0.012110463, -0.007875442, -0.014343254, 0.0036859252, 0.015059203, -0.0071291556, 0.0062372526, -0.01674593, 0.0074021867, 0.027521573, -0.004168281, 0.003139862, 0.05412091, -0.05844087, -0.019451976, 0.010848451, -0.0028289095, -0.008991837, -0.022315774, 0.017012896, 0.0073718503, 0.015277629, 0.029390322, -0.024657777, 0.011085078, 0.0012369843, -0.029414592, 0.051111493, 0.0036616556, 0.010356994, -0.012547313, 0.011606872, 0.019925231, -0.0033552535, 0.022206562, -0.019257821, 0.005421192, -0.003886148, -0.056499317, -0.019961635, -0.021017358, 0.0058580423, 0.0020416689, -0.0058368063, 0.045141205, -0.042325947, -0.024815528, 0.02999706, -0.0058004023, 0.0077965655, 0.00999902, 0.049242746, 0.014707296, -0.014318985, -0.026405178, -0.026720682, 0.013942808, -0.0024284634, -0.020265004, -0.00930734, -0.0072262334, 0.001512291, -0.0024588, -0.011006203, -0.04218033, 0.029244706, 0.026016867, -0.006173545, 0.0069774715, -0.009216329, 0.023541382, -0.019027261, 0.008761277, 0.010617891, 0.003801205, 0.060139734, -0.045602325, 0.008749142, 0.015059203, -0.017061435, 0.03485095, -0.011254964, 0.025070358, 0.002844078, 0.027230341, -0.009786662, -0.02764292, -0.02905055, -0.0017231321, 0.024682047, 0.014270445, -0.044752896, -0.019015126, -0.006443543, -0.012104396, 0.019172877, -0.04914567, 0.047543883, 0.007305109, 0.0076388144, 0.005132992, -0.021162974, 0.03232693, 0.028832125, 0.012790008, 0.005005577, -0.034365565, 0.008372965, -0.022534199, -0.043029763, 0.022631276, 0.025094626, 0.013518092, -0.047471073, -0.045165475, -0.010174973, 0.011042607, -0.0066498336, 0.020447025, -0.0015244258, -0.02358992, -0.01204979, 0.012559448, 0.004623333, -0.032448277, 0.010193176, 0.039243724, 0.024572834, -0.026186753, -0.053295746, 0.0013719832, 0.03043391, 0.013190455, 0.014039886, 0.029317515, 0.0286137, -0.014561679, 0.010799912, -0.020325677, -0.011885971, 0.0013530226, -0.030458178, 0.004171314, 0.017825922, -0.029171897, -0.019124338, -0.01246237, -0.016321216, -0.0057215267, -0.025167435, -0.017983673, 0.005427259, -0.0001678954, 0.043879192, -0.008664199, 0.031501766, 0.02994852, 0.007566006, -0.007189829, -0.016927952, -0.026259562, -0.023638459, -0.036913857, 0.032472543, -0.010253849, -0.023917558, -0.021284321, 0.00025274372, -0.03730217, -0.017522555, -0.049218476, -0.027230341, 0.0041258093, -0.0053483834, -0.005045015, -0.0041895164, 0.004383672, -0.029147629, 0.027157532, -0.021599824, 0.03089503, -0.021672633, -0.009149589, 0.02448789, -0.018990858, -0.049218476, 0.0130205685, -0.017838057, 0.015423245, -0.037326436, -0.00748713, 0.037035204, 0.021393534, 0.008263753, 7.863875e-05, -0.008203079, -0.00999902, 0.012425967, -0.023407899, 0.019573323, 0.011315638, 0.028686509, -0.021442072, 0.024851933, -0.024754854, 0.0182021, -0.02900201, -2.2823726e-05, 0.019427707, 0.019779615, -0.0031095252, -0.042423025, -0.034923762, -0.018893778, 0.0003939996, -0.014100559, -0.0012756637, -0.04497132, -0.030628065, 0.012632256, -0.004650636, -0.02587125, 0.015326167, 0.011916308, 0.010520813, -0.029778633, -0.02548294, 0.020835336, -0.022861836, -0.025798442, -0.01191024, 0.007996789, -0.011764623, 0.057373017, -0.0009229981, -0.0076934206, 0.026162483, -0.00251644, -0.004702209, 0.005430293, -0.014852913, 0.0008130271, 0.01248664, -0.019415572, 0.00056312745, -0.008160608, 0.011631141, -0.03189008, 0.015568862, 0.006922865, -0.02997279, -0.038418565, 0.01216507, 0.008979702, -0.010290253, -0.04778658, -0.014962126, 0.028006963, -0.0037556998, -0.01112755, 0.025604285, 0.035384882, 0.031040646, 0.02292251, 0.013918539, 0.008306225, -0.0053180465, -0.013226858, 0.003130761, 0.019136474, -0.011685748, -0.009865537, 0.003810306, -0.0135909, -0.019816019, -0.020811066, 0.013336071, 0.022048809, 0.023517111, 0.004705243, 0.014755835, 0.027133262, 0.015678074, -0.00817881, 0.020143656, 0.017850192, 0.05082026, -0.011570468, -0.0041864826, 0.05392675, 0.041452248, -0.0053392826, 0.0036222178, 0.027788538, 0.016503237, 0.010344859, 0.021466343, -0.026696412, -0.031987157, -0.016491102, -0.030385371, -0.0077601615, 0.008530717, 0.019816019, -0.022740489, -0.0122682145, -0.018214233, -1.7976943e-05, 0.018517602, 0.017813787, -0.027375957, -0.01933063, 0.0049600718, 0.012189339, 0.011879903, 0.0075478037, -0.032496814, -0.024427217, -0.038345754, -0.0058459076, -0.01584796, -0.017231321, 0.00070950267, 0.017983673, 0.012620122, -0.043782115, 0.00011196187, 0.009810931, -0.033855904, 0.022582738, 0.019937366, 0.02727888, 0.003112559, 0.0023298687, -0.018748162, -0.013882134, 0.0012316754, 0.00089569495, 0.028395275, -0.0034826682, 0.012705065, 0.019160744, 0.035263535, -0.0023131834, -0.020507699, 0.011327773, 0.005181531], "content_hash": "ea510f95fe6b84bfc72d56660e0d67e8"}, {"content": "on the left to tokens on the right. Thicker lines mean stronger attention. **Navigating BertViz**: Single-click a head square to select/deselect it. Double-click to show only that head. Hover over tokens or lines to see exact attention weights. ## Stage 4: MLP (Feed-Forward) **Icon**: Network | **Summary shows**: \"X layers\"\n\nThis stage shows the expand-then-compress pattern of the feed-forward network:\n- Input dimension (e.g., 768) \u2192 Expanded dimension (e.g., 3072, which is 4x larger) \u2192 Back to input dimension\n- An explanation of why this expansion matters for storing factual knowledge\n- The total number of layers in the model\n\n## Stage 5: Output Selection\n\n**Icon**: Bullseye | **Summary shows**: \"\u2192 [predicted token]\"\n\nThis stage reveals the model's prediction:\n- Your full prompt with the **predicted next token** highlighted\n- A **confidence percentage** for the top prediction\n- A **top-5 bar chart** showing the five most likely next tokens and their probabilities\n- A note about how beam search and other techniques can influence the final output", "source_file": "pipeline_stages.md", "chunk_index": 1, "embedding": [0.03332839, 0.013307699, -0.00044659345, -0.024663603, -0.012553596, -0.00910838, 0.0111341085, 0.047552846, -0.029661383, 0.06736653, 0.025624715, 0.0034452155, -0.03329882, -0.03536891, 0.0016246482, -0.0106018, -0.004964511, 0.0138474, 0.00047593508, 0.0392725, 0.0029184523, -0.023569414, 0.061629433, 0.02918822, 0.03628566, 0.04225934, 0.008420817, 0.024293944, 0.04551233, 0.01445364, -0.019133514, -0.03131745, -0.005522695, -0.06293063, -0.007696286, -0.02809403, -0.016309325, 0.035014033, -0.029055143, 0.036137797, -0.013492528, -0.0056483787, -0.05260977, 0.022149926, 0.012664493, 0.040662415, -0.0035986237, -0.023451123, -0.03504361, 0.063758664, -0.030134546, 0.046931818, -0.015821375, 0.0056816484, -0.013581246, 0.004343485, -0.0063174604, -0.019902403, 0.007252696, -0.021987276, 0.024737533, -0.00664276, 0.038651474, 0.028522834, 0.015170777, -0.013529494, -0.09114295, 0.018645566, 0.015747445, 0.0077997902, 0.057016097, -0.0060180374, -0.004398934, 0.027768733, 0.03108087, 0.010047313, 0.018438557, 0.021795053, -0.0033269248, -0.04427028, 0.030578135, -0.024974115, -0.010668339, -0.07824927, 0.0032751726, -0.05651336, -0.038030446, -0.045423612, -0.049001906, -0.054975584, -0.013078511, -0.012191331, -0.050450966, 0.007814577, 0.010823595, 0.012901075, 0.0155995805, -0.009500219, -0.003277021, 0.0036429826, 0.032914374, -0.007895902, -0.040721558, -0.04237763, 0.032382067, -0.01613189, 0.022490012, 0.024249585, -0.024929756, -0.024086935, -0.08889543, -0.006206563, -0.011148894, 0.056424644, 0.021735908, -0.0148011185, 0.00750776, -0.0059182295, 0.046103783, 0.011060176, -0.017906249, -0.017906249, 0.001033195, 0.02475232, 8.1324826e-05, 0.014283597, 0.032145485, -0.0098624835, 0.007463401, -0.025299415, -0.016767701, -0.0021588046, 0.036995403, -0.030060614, -0.021307105, -0.0152003495, -0.028330613, 0.022652661, -0.04296908, 0.044447716, -0.008820048, -0.0031051298, 0.0039072884, -0.019488387, -0.049770795, 0.02164719, -0.04607421, 0.0553896, -0.053556096, 0.015998812, -0.005977375, 0.0016477519, -0.019562317, -0.03430429, -0.054295413, -0.044151988, -0.01239834, 0.022682235, -0.0128493225, -0.021676764, 0.015052486, 0.0048905793, 0.004088421, -0.0044654724, -0.020893088, 0.06103798, -0.03510275, 0.021410609, -0.03983438, -0.06364037, 0.00039045158, 0.020597361, -0.011126715, -0.031790614, -0.0036928866, -0.00086823496, -0.056661226, -0.051308572, -0.017595736, -0.042141046, -0.010941886, 0.0051715197, -0.016516333, -0.021765482, -0.031908903, 0.010017741, -0.013093297, 0.03785301, -0.021987276, -0.01398787, -0.004095814, 0.03596036, 0.063758664, -0.009256244, -0.011902997, -0.023702491, 0.030785143, -0.0035912304, 0.020656506, -0.00039645855, 0.018601205, 0.0658879, 0.051900025, -0.024500953, 0.03850361, 0.013551674, -0.013825221, -0.01076445, -0.012790177, 0.0021310803, 0.018630778, -0.02757651, 0.045541905, 0.007322931, -0.06985063, 0.024293944, -0.069732346, 0.050894555, 0.009086201, 0.029410016, 0.030607708, 0.013440776, 0.035782926, -0.00024466758, -0.04959336, 0.038681045, 0.0471684, -0.035871644, -0.0070715635, -0.006579918, 0.013913939, 0.030430272, 0.011259792, 0.0048610065, -0.0430578, -0.0440337, -0.014645862, -0.05680909, 0.031672325, -0.06364037, -0.0019037403, 0.024441807, 0.028877707, 0.015377786, 0.0118142795, 0.023451123, -0.01580659, -0.017994966, 0.039390787, 0.02790181, 0.051870454, 0.0063359435, -0.029927537, -0.0011062025, 0.0035283884, -0.012339194, -0.029498734, 0.034895744, 0.0006492124, -0.00014670815, -0.0032160273, -0.011392869, -0.025816936, -0.031228734, -0.03773472, -0.019724967, -0.020390352, -0.0020368174, -0.059174903, -0.047079682, -0.0041956217, 0.007271179, 0.022061208, 0.017285222, -0.052905496, 0.010128638, -0.013795648, 0.044980023, -0.019961549, 0.0048018615, -0.044211134, -0.019089155, 0.058879174, 0.026304886, -0.010786629, -0.017240863, -0.009988167, -0.0070013283, 0.025506424, -0.0011607271, 0.03672925, -0.029705742, 0.033564974, -0.0023251506, -0.018128043, -0.0012050861, 0.05269849, 0.012797571, 0.0010618435, -0.007511457, 0.04341267, -0.045778483, -0.00096480816, -0.007430132, -0.0015738202, 0.028404545, 0.033890273, 0.04264378, 0.0035672027, -0.01207304, -0.019621463, -0.018808214, 0.012583169, -0.03297352, -0.033801556, -0.009389321, 0.049652506, -0.004679874, 0.024767106, -0.002339937, 0.009810732, -0.02096702, -0.014098768, 0.016797274, 0.026216168, -0.025979586, -0.03462959, 0.033505827, 0.024486165, -0.022164712, 0.022223858, 0.0054894257, -0.030104972, 0.038769763, -0.030785143, -0.045896776, -0.024974115, 0.02342155, -0.009906842, 0.051515583, -0.03820788, -0.036137797, -0.024707962, -0.018689925, -0.037438992, 0.029927537, 0.015126418, -0.0043065194, 0.01475676, 0.020212917, -0.024367876, 0.0010960369, 0.02967617, -0.033771984, 0.01765488, -0.007285965, 0.001109899, -0.019562317, 0.00051752164, 0.03232292, -0.028892493, 0.015229923, 0.03962737, 0.028522834, -0.014010049, -0.023436336, 0.0014204121, -0.002003548, -0.0059551955, -0.03442258, -0.0066390634, -0.016057957, 0.012538809, -0.010838382, -0.03995267, -0.01926659, -0.005337866, -0.0146015035, -0.048676606, 0.026023945, -0.017019069, -0.04959336, -0.028522834, 0.009648082, 0.021247959, 0.006195473, 0.017640095, -0.008642611, -0.054413702, 0.0066390634, 0.017462658, -0.02479668, 0.00020689314, -0.011784707, 0.035575915, 0.014505392, -0.028182749, 0.038888052, 0.04196361, 0.069968924, 0.039242927, -0.032027196, -0.06914089, -0.016279751, -0.0018556847, 0.018349838, 0.027753945, 0.01048351, 0.02204642, 0.0130489385, 0.036137797, -0.023865141, 0.012309622, -0.012509237, -0.014963768, 0.055212166, 0.031731468, 0.01310069, -0.0012143275, 0.07369508, -0.01342599, -0.033831127, -0.042909935, -0.020937447, -0.023569414, -0.014815905, 0.05935234, -0.033505827, 0.028670698, -0.0130489385, 0.0014629228, 0.058435585, 0.0073266276, 0.018320266, 0.009810732, -0.05580362, 0.013174622, 0.037616428, 0.0070272046, -0.027310356, 0.0030478328, -0.02806446, -0.023805995, -0.03596036, -0.03643352, 0.017270437, -0.056749944, -0.06026909, 0.010002954, -0.017876675, -0.020257276, 0.04160874, -0.0030552258, 0.09587458, -0.02895164, -0.05024396, 0.09132039, -0.0016773245, 0.047257118, -0.0007753583, 0.043235235, 0.004310216, -0.020626934, 0.024767106, -0.006853465, 0.0104169715, 0.023229329, -0.01221351, 0.0708561, 0.0116959885, -0.0035505681, -0.022830097, 0.012169152, -0.021247959, 0.017728813, -0.004812951, 0.023081465, -0.014838085, 0.020272061, -0.01713736, 0.029897964, 0.015067273, -0.0028445206, -0.010734878, 0.0052084858, 0.08759423, -0.0039701303, -0.013699537, 0.050450966, -0.0012032378, -0.010224748, -0.023554627, 0.030489417, -0.019325737, 0.033003096, -0.06387696, -0.032234203, -0.031908903, 0.008945731, -0.03507318, 0.046636093, -0.014875051, -0.010076885, 0.045660194, 0.023643345, 0.0019832167, 0.03539848, 0.06488243, 0.001673628, 0.054768573, -0.05269849, -0.020937447, -0.021233173, 0.018985651, -0.029395228, 0.01564394, 0.032677796, 0.017196504, -0.006250922, 0.039656945, 0.006605794, -0.00012707004, -0.0020368174, 0.012058254, -0.0043323953, -0.0079106875, -0.016235393, -0.018512487, -0.046103783, -0.024264371, 0.0070087216, -0.051900025, 0.004690964, 0.0022789435, -0.05107199, 0.012435306, -0.022756165, -0.0036318928, -0.02031642, 0.011311544, 0.0074042557, 0.017847104, -0.00154702, 0.02092266, -0.00073931663, 0.0060586995, 0.0021772874, 0.015777016, 0.0155995805, 0.018335052, 0.006021734, -0.017566163, 0.015510863, -0.01821676, -0.023111038, -0.013943511, -0.03584207, -0.02895164, 0.0074042557, -0.03273694, 0.015658725, -0.009137954, -0.006213956, -0.019399667, -0.032559503, 0.019000437, 0.021957703, -0.0075853886, 0.0074412217, -0.020020694, -0.029040357, -0.023155397, 0.052994214, 0.027058989, -0.0061141485, -0.01821676, 0.010254322, 0.02608309, -0.0038407499, 0.022652661, -0.031140016, -0.036463097, -0.0031679717, -0.02407215, -0.013803042, -0.03108087, -0.010609194, -0.0065910076, -0.02584651, 0.006365516, 0.022667447, -0.021661976, -0.01664941, 0.027561724, -0.031701896, -0.010202569, 0.0050125667, -0.012701459, -0.022445653, -0.0010683126, 0.001946251, -0.020065052, -0.018970864, 0.017625308, -0.0420819, 0.037646003, -0.04092857, -0.0070863497, 0.015303854, 0.017965393, -0.04882447, -0.045305323, -0.019281378, -0.01923702, -0.026881551, -0.013995264, -0.041549593, -0.0043139122, 0.013019365, -0.0132485535, -0.0033102902, 0.009507611, 0.001470316, 0.013743896, 0.00034008565, 0.030164117, 0.002522918, -0.009692441, -0.011474194, -0.013507315, -0.033890273, 0.0016052412, 0.028522834, 0.012760605, -0.027147707, -0.0028796382, 0.021011379, 0.008376457, -0.0049792975, -0.01713736, 0.014387101, 0.0076593203, -0.027709587, -0.023510268, -0.015274282, 0.0026097877, -0.004103207, -0.005929319, -0.013174622, -0.028020099, -0.03294395, -0.036877114, 0.03023805, -0.014424067, -0.026186595, -0.04373797, 0.005818422, 0.010261714, -0.018926505, 0.014941589, -0.013026759, 0.009204492, -0.017344369, -0.04639951, -0.017285222, 0.042229764, 0.014401888, 0.034481727, 0.018231548, -0.016885992, -0.08593816, 0.02552121, -0.01753659, 0.020360779, 0.016945137, -0.036019504, 0.00019557234, -0.03915421, -0.026275313, 0.041786175, 0.0029979288, 0.002643057, -0.058760885, -0.0018981954, 0.010002954, -0.008997483, 0.031376597, -0.025255056, 0.013226374, 0.0074523115, -0.005762973, -0.0045098313, 0.022549156, -0.015880521, -0.021602832, 0.0123318015, 0.0138843665, 0.06529644, -0.009803338, -0.048883613, -0.00032298896, 0.026585825, -0.029587451, -0.032145485, -0.01963625, -0.028685484, -0.032589078, 0.046576947, 0.008043765, -0.01032086, -0.04261421, 0.0020405138, -0.0133668445, -0.011008425, -0.016338898, 0.029720528, -0.011673809, -0.007330324, 0.0018288845, -0.014461033, -0.032796085, 0.03326925, -0.035014033, 0.0007961516, 0.013832614, -0.010357826, -0.024175653, -0.028552407, -0.0075484226, 0.020907873, 0.0215289, 0.035250615, 0.00080908963, -0.033919845, -0.015688298, 0.035250615, -0.029735314, -0.006901521, -0.041194722, -0.040337116, -0.015703086, -0.009004876, -0.0118142795, 0.01942924, 0.006764747, 0.022460438, -0.026334457, 0.037764292, -0.035487197, -0.034659162, -0.007792397, 0.014490605, -0.017107787, 0.012583169, 0.0065725246, 0.01398787, 0.018335052, -0.043442246, 0.033210102, -0.004044062, 0.0044285064, -0.009234064, -0.01802454, 0.021661976, 0.015022914, -0.0142022725, 0.018985651, -0.0048166476, 0.024929756, 0.004376754, 0.0027724374, -0.028389757, 0.02479668, -0.021070523, 0.06571046, -0.020375567, 0.03528019, -0.020893088, -0.016664196, 0.0035838373, -0.0146015035, -0.020893088, -0.003759425, -0.022223858, 0.010402185, -0.0039331643, -0.010076885, -0.010202569, 0.0031143713, -0.029853605, -0.032145485, -0.0154812895, 0.0059367125, -0.078426704, 0.0039553437, 0.018320266, 0.010136031, -0.113085866, -0.045453187, 0.033505827, -0.019296164, -0.042022757, -0.019414455, 0.040751133, -0.003125461, -0.020227702, 0.002541401, 0.0038407499, -0.012923255, -0.023879927, -3.9420593e-05, 0.0003608789, -0.041490447, 0.024116509, 0.0027872236, -0.0040366687, -0.03184976, -0.038030446, 0.0056816484, -0.0016828694, 0.024648815, -0.023835568, 0.032027196, -0.048676606, -0.004502438, 0.011052784, 0.009707227, -0.0134851355, 0.024338303, 0.017921034, 0.046103783, 0.043205664, -0.011141501, -0.01576223, 0.009004876, 0.022667447, 0.009219279, 0.003785301, -0.036817968, 0.0122504765, -0.04897233, 0.036788397, -0.007999406, -0.028212322, -0.043116946, 0.019251805, 0.019813685, -0.02104095, -0.012339194, -0.008738723, -0.01108975, 0.011599878, 0.0300754, -0.004794468, -0.016723342, 0.0006492124, 0.021277532, -0.022401294, 0.030933008, 0.033771984, -0.022297788, 0.025920441, -0.017832316, 0.01120804, 0.02463403, -0.014948982, -0.010409578, 0.018349838, -0.005371135, 0.00031790615, -0.023465909, 0.02180984, 0.014505392, 0.011703382, 0.004775985, -0.0027927684, -0.010794022, 0.05172259, -0.021795053, -0.017566163, 0.01447582, 0.012553596, 0.020774797, 0.006091969, -0.028655913, 0.041431304, 0.02309625, 0.013078511, -0.0020516035, -0.005689041, 0.013078511, -0.022918815, -0.014616289, 0.035546344, 0.02657104, 0.048351306, -0.009722013, 0.07002807, 0.026541466, -0.028448904, -0.054265838, -0.038769763, -0.006827589, 0.0013963842, 0.0026097877, 0.015333426, 0.00030889572, 0.014438854, -0.01328552, 0.01959189, 0.010956672, -0.034156427, 0.03406771, 0.03616737, 0.028212322, 0.00013769772, -0.021928132, 0.0028075548, 0.0142022725, -0.022963174, 0.019458814, -0.014778939, -0.008871799, 0.009936416, 0.0046392116, -0.0029591147, 0.0064653237, -0.021868985, 0.013662571, -0.01440928, 0.0013270733, -0.013204195, 0.017477445, 0.0017503321, 0.027058989, -0.03882891, 0.011341117, -0.0054487633, 0.015673513, 0.01907437, 0.03241164, -0.034925316, 0.017447872, -0.015111632, -0.020538216, 0.0021366251, 0.02302232, 0.052195754, 0.007570602, 0.017507017, -0.019887617, 0.0020201826, 0.03720241, 0.020582575, -0.024929756, 0.0032215721, 0.05760755, 0.0009527943, 0.051486008, 0.049889084, -0.023820782, 0.008391243, -0.008716543, -0.005611413, 0.008642611, -0.026304886, 0.039568227, -0.041786175, -0.007537333, 0.032796085, 0.031228734, 0.021617617, 0.03332839, -0.0060771825, 0.04483216, 0.0011163681, 0.03874019, -0.0027188368, 0.032382067, -0.018113257, 0.005175216, 0.043146517, -0.0102986805, 0.0016958074, 0.028079245, -0.0057851523, -0.0075336364, -3.7514543e-05, 0.019281378, -0.019769326, 0.022889242, -0.0235842, -0.006472717, 0.010343039, 0.0046946607, -0.007016115, 0.020789584, -0.04639951, 0.012243083, 0.028345399, 0.0042880364, -0.0032419034, 0.00796244, -0.0023251506, -0.0006690815, 0.020168558, 0.023968644, -0.021307105, 0.0054672463, -0.018408984, 0.01891172, -0.037438992, 0.016605051, 0.019931976, -0.0042436775, 0.026437962, -0.008450389, -0.034156427, -0.04882447, -0.012176544, 0.014564537, 0.0038296601, -0.0055189985, 0.037468567, -7.964981e-05, -0.01625018, 0.0021144454, -0.006679726, 0.013499921, -0.006605794, 0.017226078, 0.009892057, 0.011725562, 0.015658725, -0.0048351306, 0.018113257, 0.0039220746, 0.046369936, 0.025299415, -0.00074902014, 0.0072748754, -0.016442401, -0.014335349, -0.006897824, 0.0054044044, -0.025535995, -0.009951201, -0.013300306, 0.017994966, 0.019059582, -0.00971462, -0.018763855, -0.014564537, 0.036315233, 0.013011972, -0.0027447129, 0.023406764, -0.0024064756, 0.03226378, 0.032382067, -0.057341397, 0.01991719, -0.0025136764, -0.020405138, -0.0063950885, 0.010550048, 0.0056335926, -0.000102291386, 0.048440024, -0.013396417, 0.025935227, 0.0029332386, 0.031879332, 0.0027151403, -0.02725121, 0.024707962, -0.01713736, 0.013581246, -0.018275907, 0.0067684436, 0.0049127587, 0.0083468845, 0.038681045, -0.01870471, -0.032382067, 0.011104535, -0.027546937, 0.015998812, 0.018231548, 0.019710181, 0.009211885, -0.006960666, 0.006561435, -0.010209963, 0.0017780565, 0.011718168, -0.03528019, 0.024678389, -0.0020774796, 0.020286849, -0.01015821, -0.013315092, 0.015895307, 0.036315233, -0.034186002, 0.006088272, -0.01106757, -0.0033343178, 0.0049090623, 0.007999406, 0.01713736, -0.020686079, -0.016634624, 0.030001469, -0.0519296, 0.0052972035, -0.022490012, 0.002885183, 0.009248851, -0.0077628247, 0.021395823, -0.014771546, -0.017107787, 0.010875347, -0.0082285935, 0.008605645, 0.0038148738, 0.018926505, -0.004905366, -0.004838827, 0.02314061, -0.025018474, -0.009943808, -0.038976774, -0.005400708, -0.0007263786, -0.014520179, 0.029439587, 0.03610822, -0.0044913483, -0.011126715, 0.021395823, 0.0007448615, 0.012206118, -0.0038851087, 0.002606091, -0.007792397, 0.07606089, -0.03362412, -0.015436931, -0.008583466, -0.0056446823, 0.013470349, -0.014978555, 0.0135664595, -0.026600612, 0.008738723, -0.0005475564, -0.0045246175, -0.0034858778, -0.02790181, 0.03471831, -0.022075994, -0.006535559, 0.004247374, 0.03962737, 0.007481884, 0.013056331, 0.030903434, -0.005275024, 0.009766373, -0.002308516, 0.025003688, 0.004761199, 0.014217058, 0.017684454, 0.027502578, -0.004410024, -0.04418156, -0.004746413, 0.051840883, -0.01580659, -0.001293804, -0.01239834, 0.033712838, -0.018039325, 0.013588639, -0.0053341696, -0.0073709865, 0.019118728, 0.001521144, -0.005352652, -0.0005244527, 0.012413126, 0.002556187, -0.009788552, -0.008879192, 0.007688893, 0.011267185, 0.00826556, -0.0034304291, 0.02092266, -0.052580196, -0.01842377, 0.007977226, -0.013573853, -0.0041771387, -0.019503172, 0.017773172, 0.008332098, 0.0009527943, 0.018660352, -0.04926806, 0.0067019053, -0.006487503, -0.01137069, 0.053999685, -0.01926659, -0.011659023, 0.009655475, -0.0087461155, 0.021336678, 0.00826556, 0.01613189, 0.0058886567, 0.015089452, -0.0027724374, -0.050539684, -0.005415494, -0.038237456, 0.0011062025, -0.010557441, 0.012760605, 0.05790328, -0.017403513, -0.026511895, 0.034807026, -0.024042577, 0.003251145, 0.022830097, 0.03596036, -0.0018335052, -0.01903001, -0.013270733, -0.019606676, -0.024131294, -0.02946916, 0.000919063, -0.002155108, 0.009943808, 0.003302897, -0.0104169715, -0.014187486, -0.06293063, -0.009522398, 0.037113693, -0.011851246, 0.0054894257, -0.005001477, 0.032027196, -0.018689925, 0.0038333565, 0.02257873, -0.017743599, 0.041106004, -0.030814717, -0.017610522, 0.011193253, -0.013854793, 0.016900778, -0.023717277, 0.017166931, 0.023776423, 0.0008844075, 0.007175068, -0.034984462, -0.014135733, -0.00022583813, 0.031761043, 0.010986245, -0.0032289652, -0.0037113694, 0.010727484, -0.0060771825, 0.004265857, -0.045127887, 0.04713883, -0.00012522175, 0.014904623, -0.00650229, -0.022918815, 0.030430272, 0.0069754524, 0.03539848, 0.007082653, -0.026068304, 0.010091672, 0.003302897, -0.045098312, 0.020493856, 0.008790474, 0.026334457, -0.042850792, -0.01074227, -0.0019702788, -0.005685345, -0.009285817, 0.025595142, -0.0010858712, -0.025240269, -0.012812357, 0.038681045, 0.010712698, -0.016398042, 0.0025746701, 0.032796085, 0.021942917, -0.015954452, -0.037024975, -0.008701757, 0.01267928, 0.0056003234, -0.008694364, 0.006010644, 0.04184532, -0.01802454, 0.017921034, -0.0047390196, -0.001673628, -0.008857013, -0.016605051, 0.005977375, 0.00546355, 0.00033107522, -0.017462658, 0.0037760597, -0.0016671589, 0.01737394, -0.032086343, -0.0095815435, -0.0046503013, 0.011082356, 0.055744473, -0.01963625, 0.0465178, 0.041697457, -0.0019111334, -0.01580659, -0.011104535, -0.011984322, -0.028522834, -0.017255649, 0.027280783, -0.032056767, 0.0048647034, 0.007999406, 0.013758683, -0.027635654, -0.007563209, -0.022061208, -0.005426584, 0.009381928, 0.011562912, -0.0041771387, 0.00559293, -0.015407358, -0.028020099, 0.0033306214, -0.010808809, 0.030119758, -0.001489723, -0.019340523, 0.011880818, -0.0019795203, -0.023998218, 0.03584207, -0.024841039, 0.005640986, -0.04060327, -0.04063284, 0.022460438, 0.007200944, 0.017625308, -0.019340523, 0.013307699, 0.015141204, 0.013840007, -0.013011972, 0.024826253, -0.003219724, 0.018778643, -0.0035727476, 0.045127887, -0.018615993, 0.023362406, -0.046015065, 0.014350136, -0.0032215721, 0.008975304, -0.039656945, -0.035782926, -0.011614664, -0.02846369, 0.012686674, 0.018113257, 0.0021828322, -0.033210102, -0.03640395, 0.005123464, 0.010128638, -0.008435602, 0.00910838, 0.016678983, 0.009226671, 0.00012071654, 0.00071020605, 0.042525493, -0.030962579, -0.003996006, -0.0040292754, 0.0039516473, 0.019562317, 0.026186595, -0.0044543827, -0.0023159094, 0.012243083, -0.016028384, 0.012694066, -0.02064172, -0.002079328, -0.015089452, 0.022223858, 0.016723342, 0.0032825659, 0.00075179257, 0.0028426724, -0.021159241, 0.015185563, 0.009847697, 0.0006284191, -0.034008563, 0.014010049, -0.0070678666, 0.004838827, -0.0465178, -0.015540435, 0.023613773, -0.009655475, -0.004228891, 0.019281378, 0.027221637, 0.013218981, 0.033919845, 0.006979149, 0.02201685, 0.012634921, -0.00971462, 0.026127448, 0.005818422, -0.019118728, -0.008509534, 0.008583466, -0.012346587, 0.00010529486, -0.009566757, -0.0108975265, 0.019872831, -0.0062546185, 0.011644237, 0.0065207724, 0.009381928, 0.02519591, -0.011030604, -0.0054894257, 0.0062694047, 0.04152002, -0.0149194095, -0.0054487633, 0.036551815, 0.04015968, 0.023406764, 0.012686674, 0.010845775, -0.0019869134, 0.008672184, 0.020952232, -0.02330326, -0.044861734, 0.0017512562, -0.023850353, -0.01576223, -0.00010696987, 0.036670104, -0.041549593, -0.028522834, -0.011082356, -0.0046872674, 0.01431317, 0.026911125, -0.035546344, -0.036877114, 0.010520475, 0.005249148, 0.019828472, 0.018970864, -0.018349838, -0.03575335, -0.031465314, -0.02197249, 0.0019758237, -0.003777908, -0.0036060167, 0.0022715502, -0.026437962, -0.04184532, 0.0026227257, 0.004502438, -0.0114668, 0.009049235, 0.038888052, 0.009226671, -0.0076593203, 0.00927103, -0.03374241, -0.009847697, -0.004280643, -0.016900778, 0.034126855, -0.0075853886, -0.0015904548, -0.007696286, 0.022992747, -0.009367142, -0.005079105, 0.021114882, -0.0041290834], "content_hash": "8cae2b97916cca7b5d8bc20345194bf8"}, {"content": "# RAG Documents\n\nThis folder contains documents used by the AI chatbot for Retrieval-Augmented Generation (RAG).\n\n## Supported File Types\n\n- `.txt` - Plain text files\n- `.md` - Markdown files\n\n## How to Add Documents\n\n1. Place your documentation files in this folder\n2. Delete `embeddings_cache.json` if it exists (to force re-indexing)\n3. The chatbot will automatically index new documents on startup\n4. Documents are chunked and embedded for semantic search\n\n## Document Inventory\n\n### Category 1: General LLM/Transformer Knowledge\n- `what_is_an_llm.md` - Neural networks, language models, next-token prediction\n- `transformer_architecture.md` - Layers, encoder/decoder, residual stream\n- `tokenization_explained.md` - Subword tokenization, BPE, token IDs\n- `embeddings_explained.md` - Lookup tables, vector spaces, positional encodings\n- `attention_mechanism.md` - Q/K/V, multi-head attention, intuitive explanations\n- `mlp_layers_explained.md` - Feed-forward networks, knowledge storage, expand-compress\n- `output_and_prediction.md` - Logits, softmax, temperature, greedy vs. sampling\n- `key_terminology.md` - Extended glossary of ML/transformer terms\n\n### Category 2: Dashboard Components\n- `dashboard_overview.md` - Layout tour, navigation, typical workflow\n- `pipeline_stages.md` - What each of the 5 pipeline stages shows\n- `ablation_panel_guide.md` - How to use the ablation experiment panel\n- `attribution_panel_guide.md` - How to use the token attribution panel\n- `beam_search_and_generation.md` - Beam search, generation controls\n- `head_categories_explained.md` - Previous-Token, Positional, BoW, Syntactic, Other\n- `model_selector_guide.md` - Choosing models, auto-detection, generation settings\n\n### Category 3: Model-Specific Documentation\n- `gpt2_overview.md` - GPT-2 architecture, why it's a good starter, variants\n- `llama_overview.md` - LLaMA/Qwen/Mistral architecture, RoPE, GQA differences\n- `opt_overview.md` - OPT architecture, comparison with GPT-2", "source_file": "README.md", "chunk_index": 0, "embedding": [-0.010244178, 0.02076404, 0.03200444, -0.027844112, -0.0121614365, -0.05633984, -0.008420901, 0.014686454, -0.0046647033, 0.025839137, 0.002006543, -0.033332735, -0.0054197023, -0.018383129, -0.010983513, 0.029673655, -0.025914323, -0.00088109326, 0.0013016684, 0.046941515, 0.044435292, -0.011954673, 0.0064535183, 0.020525947, -0.026991997, -0.0069986214, 0.008264263, 0.03563846, -0.02644063, -0.02236802, 0.021929432, -0.023107355, -0.008571275, -0.022555986, -0.03077639, 0.022342958, 0.021490844, 0.019348023, 0.0014340281, -0.013646372, -0.004824475, -0.05212939, -0.016716493, 0.03984891, -0.025814073, 0.009843182, 0.011315587, 0.006923435, -0.0020159413, 0.010256709, -0.018019726, -0.013934587, -0.019222714, 0.004238646, 0.006459784, 0.011735379, -0.052179515, -0.031528257, -0.0022336692, -0.022543456, 0.0021083583, -0.008646461, 0.018934498, 0.016378153, -0.0004182256, -0.040249906, -0.08801847, 0.022806609, -0.039122105, -9.4766474e-05, 0.0663146, -0.02553839, -0.03030021, -0.015137574, -0.008364512, 0.0036308873, -0.031503197, 0.034711156, -0.02781905, -0.032831494, -0.0069735595, 0.019962048, -0.027844112, -0.035613395, -0.022906858, -0.009103847, -0.023545943, -0.006754265, -0.044259857, -0.027092246, -0.06004905, 0.012800522, -0.03837024, 0.012330607, 0.012687743, -0.00919783, -0.031052075, -0.021378063, -0.0039128372, 0.016779147, 0.003181334, -0.05448524, 0.008220404, 0.0028508261, 0.050099354, 0.012650149, 0.0067229373, -0.019473335, -0.013420812, -0.05177852, -0.098243855, -0.022305366, 0.0033739996, 0.02553839, 0.01362131, -0.011986, -0.02032545, 0.008715383, 0.0061840997, -0.0114158355, -0.05949768, 0.0018452051, 0.011083761, -0.024034657, 0.016265374, -0.00297927, 0.008364512, -0.020337982, -0.053683247, 0.02476146, 0.024773993, 0.022681298, 0.02669125, -0.064459994, -0.041277457, -0.023320384, -0.027768927, 0.025563452, -0.05197902, 0.021816652, 0.030701205, -0.021139972, -0.033783857, 0.044460356, -0.07192854, -0.006124577, -0.011534881, -0.0014308954, -0.03591414, 0.0038282522, -0.06320689, 0.003063855, -0.004708562, -0.010081273, -0.008746711, -0.026039634, 0.006415925, 0.042405255, 0.01283185, -0.056790963, -0.0041289986, 0.04202932, 0.006024328, -0.010876998, -0.006848248, 0.012117578, -0.0019736488, -0.012794257, -0.031628504, -0.0250998, 0.00811389, -0.046816204, -0.022443207, -0.033107176, -0.01725533, -0.007650239, -0.036691073, -0.029022036, 0.008878287, 0.01026924, -0.026089758, 0.0025093534, -0.06024955, -0.016741555, -0.041152146, -0.02669125, 0.01487442, -0.009724136, -0.027518304, 0.025400547, 0.014598736, 0.023257729, 0.067467466, 0.0041477955, 0.016591182, 0.001146596, 0.038119618, -0.031402946, 0.046064336, -0.020989599, -0.0065537672, 0.008590071, 0.01248098, -0.031603444, -0.03260593, 0.003593294, -0.04152808, 0.03621489, -0.039798785, -0.06345751, 0.005297524, -0.015062387, 0.02145325, 0.040751148, -0.07889583, -0.023345446, -0.06050017, 0.04691645, -0.0022289702, -0.0031437406, -0.002329219, 0.004630243, 0.0204257, 0.023019638, -0.0031186785, 0.035838958, -0.0053445157, -0.06345751, 0.04593903, 0.004182256, 0.025262706, 0.038620863, -0.030525768, 0.005758042, 0.06165303, -0.038846422, -0.011440897, -0.0060838507, -0.0017512217, 0.013145128, 0.036415387, 0.006253021, 0.004182256, 0.008940943, 0.027167434, -0.009999821, -0.04032509, -0.053983994, 0.07393351, 0.0151876975, 0.042279944, -0.0033238754, 0.04563828, -0.0023683787, -0.010883263, 0.020074828, -0.035513148, 0.01270654, -0.054535363, 0.00976173, -0.030225022, 0.05413437, -0.043783676, 0.0040788744, -0.0027599756, 0.01532554, -0.032229997, 0.0051941425, -0.0408514, -0.018358067, -0.035738707, 0.019799145, 0.01884678, 0.01759367, -0.06626448, -0.0066101574, -0.024736399, -0.006190365, -0.0022884929, 0.054234616, 0.018583627, -0.00025864982, 0.009623888, 0.047066826, -0.0042731063, 0.04190401, 0.036841445, 0.017117487, 0.005566943, -0.02634038, 0.0091414405, -0.003815721, 0.037743684, -0.049347486, -0.018370599, 0.020175077, 0.061101664, 0.021027192, 0.03268112, 0.015488444, -0.0031139792, -0.009348203, 0.017067363, -0.00049223745, 0.025237642, 0.045137037, -0.019949518, -0.039097045, 0.0102755055, -0.03258087, 0.003045058, -0.02395947, -0.005385242, -0.04187895, -0.026991997, -0.013633841, 0.045563094, 0.0352124, 0.050801095, -0.011271728, -0.008226669, -0.024773993, 0.055237107, -0.042756125, -0.04744276, -0.014686454, -0.0123431375, 0.031277634, -0.052029144, -0.01908487, 0.07047493, -0.022894327, 0.021490844, 0.03686651, -0.0147992335, -0.036365263, 0.0032894148, 0.044560604, -0.05050035, -0.0034899125, -0.030400459, -0.009661481, -0.046841267, 0.0049121925, -0.0333578, 0.0045519234, -0.014034837, 0.06656522, 0.006979825, 0.01973649, 0.0015444586, 0.013947119, -0.014849358, -0.011434632, -0.03192925, 0.013007286, -0.04937255, -0.02872129, -0.009567497, 0.004182256, -0.030250086, 0.012518573, 0.035613395, 0.006249888, 0.01431052, -0.016353091, -0.022029681, 0.0057549095, 0.037743684, -0.014824295, -0.011121355, -0.024586026, -0.021302877, 0.0014833694, -0.06756771, -0.0052787275, -0.024134906, 0.00029271876, -0.01441077, 0.05393387, -0.015914502, -0.020726446, -0.0019141261, -0.006321942, 0.037969243, -0.0017841159, 0.000597577, 0.011992266, -0.0073745544, 0.0622044, -0.039447915, 0.006256154, -0.034109663, -0.016854335, 0.009930899, 0.032505684, -0.025149925, -0.03192925, 0.032856554, -0.0062843487, 0.04052559, -0.013558654, -0.039097045, 0.002075464, 0.0181701, 0.027844112, 0.061151788, 0.059998926, 0.006923435, -0.03826999, 0.03699182, -0.013358157, 0.0032142282, -0.012744132, -0.02008736, 0.014022306, 0.03200444, 0.054635614, 0.0040506795, 0.008358247, -0.0076251766, -0.06731709, -0.03974866, -0.01918512, -0.00380319, 0.024109844, 0.056440093, -0.0039911564, 0.020112421, -0.041503016, -0.025814073, -0.0146112675, 0.041152146, -0.01277546, -0.010519861, -0.023758972, 0.029924275, 0.025325362, -0.003536904, -0.011503553, 0.03313224, -0.013546123, -0.030550832, -0.013884463, -0.048345, -0.020563541, -0.02781905, -0.015313009, 0.0047743507, -0.03225506, -0.013746621, 0.008264263, -0.0023401836, -0.0646605, -0.0026377973, -0.040024344, 0.05934731, 0.013583717, 0.056690715, -0.013834339, 0.045337535, 0.03328261, -0.021766527, 0.016503464, -0.003137475, -0.041578203, -0.051051717, -0.0055136858, 0.020713914, -0.032054562, -0.014598736, -0.034310162, -0.040475465, -0.015588693, -0.027994486, -0.040826336, -0.016140062, -0.04904674, 0.039122105, -0.025124863, 0.03543796, -0.0023025903, -0.017894417, -0.016929522, 0.010996044, 0.09889547, 0.00624049, -0.012267951, 0.050876282, -0.003145307, -0.032731242, -0.014210272, 0.027042123, -0.012957161, 0.034836467, 0.0041164677, 0.009617622, -0.098243855, 0.0016259108, -0.023545943, -0.031152325, -8.752193e-05, 0.0031969978, 0.041728575, -0.020275326, 0.008214138, -0.028245108, 0.016741555, 0.0047461553, 0.011766707, -0.00754999, -0.033934228, 0.011666458, 0.03192925, 0.018220225, 0.015651349, -0.00093278405, 0.027317807, -0.011021106, 0.0015256619, 0.0080136405, -0.027944362, 0.010569986, -0.006923435, 0.0047712177, -0.003593294, -0.0033238754, -0.066464975, -0.023608599, 0.023997065, 0.038846422, -0.037618373, 0.01825782, 0.00129227, -0.03962335, -0.0018718336, -0.01532554, 0.023658725, -0.0049654497, -0.013658904, -0.014886951, 0.02634038, 0.03493672, -0.020475823, 0.007650239, -0.0001572262, -0.00042253316, -0.0010573119, 0.014849358, 0.027844112, 0.050700847, -0.006848248, 0.03721738, -0.04120227, 0.005779972, -0.008427167, 0.016039813, -0.03781887, 0.031854067, -0.014749109, 0.0025532125, -0.025275236, -0.022844203, 0.007731691, -0.019586116, 0.04062584, -0.0066602817, 0.00035811545, 0.0014301122, -0.007468538, 0.019059809, 0.008940943, -0.0011027372, 0.023195073, 0.038620863, -0.026064696, -0.013997243, -0.016140062, -0.017305454, 0.014661391, -0.029272659, -0.039523102, -0.03769356, 0.012455917, -0.03646551, 0.018997153, 0.028595978, 0.015538569, 0.0036026924, -0.005595138, 0.05413437, 0.006547502, -0.0063845976, -0.02476146, -0.013721558, -0.013495998, -0.00064065267, 0.00085446466, -0.028470667, 0.02395947, 0.0067354683, -0.0005952274, -0.03531265, 0.007988579, 0.0121614365, -0.017180143, -0.028570917, -0.00043623906, -0.016904458, 0.009799323, -0.010714094, -0.01021285, -0.039322603, -0.05142765, -0.017167613, 0.008032437, 0.009329407, 0.011284259, 0.006823186, -0.021603623, -0.0089472085, -0.00028175404, 0.014210272, -0.015764128, 0.016954584, 0.047041763, -0.02235549, -0.05130234, -0.04280625, -0.016729023, -0.032380372, 0.02418503, 0.049146987, -0.015125043, -0.027768927, 0.016177654, -0.00026609015, -0.02145325, 0.023671255, -0.004219849, 0.024134906, 0.024773993, -0.00090850506, -0.021152504, 0.045212224, -0.03576377, -0.007631442, 0.016202718, -0.005830096, 0.01169152, -0.027267681, -0.036014393, -0.0042574424, 0.008878287, 0.04210451, -0.021979555, 0.024134906, -0.014636329, 0.008514885, 0.05110184, -0.037768748, 0.008884553, -0.007844471, -0.018520972, -0.019962048, 0.08656487, -0.013771683, 0.02576395, -0.008646461, 0.014172679, -0.036014393, -0.031603444, 0.006547502, -0.0013337793, 0.01771898, 0.00070761575, -0.01408496, 0.011365711, 0.034560785, 0.012267951, -0.003662215, -0.025375485, -0.040901523, -0.042305008, 0.033407923, -0.049773544, 0.051828645, -0.007205385, 0.012412058, 0.0073620235, 0.009166502, -0.012650149, -0.025638638, -0.0125373695, -0.02611482, 0.07814396, 0.0049435203, 0.0035055764, 0.0044422764, -0.07217916, -0.011321852, 0.0007315032, -0.0063689337, -0.0026973202, -0.023896815, -0.009567497, -0.0059648054, 0.006654016, 0.01622778, -0.070675425, -0.025626108, 0.007850736, 0.006287481, 0.013771683, -0.014473425, -0.00065866613, -0.026064696, 0.013044879, 0.007744222, -0.03631514, -0.03531265, 0.026666189, -0.026841624, -0.0112404, -0.028445605, -0.03235531, -0.019661302, 0.0026158679, -0.047492884, 0.0013760718, 0.005551279, 0.03591414, -0.0267915, 0.006249888, 0.0015233123, 0.020601135, -0.053332377, 0.009561232, -0.031803943, -0.009166502, -0.00068137876, -0.028545855, -0.018094914, 0.020124953, -0.006406527, -0.018796656, -0.00020088926, 0.032906678, -0.02872129, 0.015751598, 8.48297e-05, 0.009529904, -0.00047970633, -0.04032509, 0.010538658, 0.032881618, 0.021077316, -0.029423032, -0.0008528982, 0.019811675, -0.021064786, 0.036941692, 0.007130198, 0.013383219, -0.025964446, -0.013458406, 0.0241725, -0.009805589, -0.00025434225, 0.029949337, -0.017769106, -0.022906858, -0.0016854334, -0.016641306, 0.06471062, -0.024084782, 0.011008575, -0.0458889, 0.01919765, 0.0011066531, -0.0027991354, -0.01691699, 0.012474714, -0.028846601, 0.002572009, 0.0044798697, -0.020839226, -0.008539948, -0.016929522, -0.02746818, -0.075587615, -0.014924545, 0.023508351, -0.0041603264, -0.005648395, -0.0048589357, 0.0029949339, -0.09187806, -0.045237284, 0.017067363, -0.008295591, 0.012493511, -0.010168991, 0.043858863, 0.032781366, 0.04030003, -0.04155314, 0.005313188, -0.00754999, 0.01384687, -0.015638817, 0.016302966, -0.042981688, 0.02032545, -0.0104634715, 0.020337982, -0.0091414405, -0.024949428, 0.012524839, 0.016252842, 0.0062686848, -0.02964859, -0.020864287, 0.005510553, 0.0068733105, 0.05573835, 0.0061590374, -0.009805589, -0.007637708, -0.04087646, 0.008063765, -0.009617622, 0.015062387, 0.008571275, -0.001727726, -0.015125043, -0.01294463, 0.011829362, -0.038645923, 0.014761641, 0.011697785, 0.02689175, 0.033583358, -0.019473335, -0.009066254, -0.0038345177, -0.005385242, 0.0075437245, 0.006306278, 0.03338286, 0.018320473, 0.04303181, -0.026816562, -0.0100875385, 0.0010549623, 0.022718892, 0.038620863, 0.00077262084, -0.014686454, -0.0033301408, 0.010526127, -0.020400638, -0.0008325352, -0.0077128946, -0.001994012, -0.058044072, -0.011202807, -0.023583537, 0.011346915, -0.0012084683, -0.0030810852, -0.005297524, 0.01963624, 0.008846959, 0.009692809, 0.017982135, -0.011459694, 0.024272747, -0.0022195717, -0.028971912, 0.030600956, 0.022342958, 0.040124595, 0.03826999, -0.035713647, -0.019799145, 0.03065108, 0.052380014, 0.009392062, 0.0051126904, -0.014523549, -0.020137483, 0.008665258, -0.028646104, -0.006472315, 0.04593903, -0.021089848, 0.03814468, 0.004542525, -0.020525947, 0.0138719315, -0.026641127, 0.02418503, -0.02373391, -0.024034657, -0.024573494, 0.01634056, 0.028194984, 0.0181701, -0.002305723, 0.014135085, -0.0050656986, 0.009160236, 0.046139523, 0.049347486, -0.00766277, -0.00039610037, 0.019109933, -0.00352124, -0.011779238, -0.011165214, -0.03363348, 0.00897227, -0.0064911116, 0.0071114013, 0.028019547, -0.0004432878, 0.01953599, 0.013383219, -0.026816562, 0.011309321, -0.034786344, 0.010970982, 0.02576395, -0.0037280035, -0.03816974, -0.020237733, -0.012881975, 0.0015389762, -0.009072519, -0.0035024434, -0.038871486, -0.004169725, -0.0091477055, 0.0020237733, -0.023408102, -0.029147347, 0.005357047, 0.007424679, -0.020789102, 0.0018843648, 0.037944183, 0.012474714, -0.009667747, -0.04278119, -0.021039722, 0.010006086, 0.0033771326, 0.01805732, 0.022769015, 0.000101913116, 0.015839316, 0.025024615, -6.0550687e-05, -0.02031292, 0.0051784785, 0.0062467554, 0.01509998, 0.027518304, 0.012631353, 0.054435115, -0.006209162, 0.043006748, 0.034786344, 0.0034993107, 0.017430766, 0.00607132, -0.02872129, 0.011735379, -0.021540968, 0.0015726535, 0.011434632, 0.012374465, -0.0061465064, 0.021039722, -0.013633841, 0.033783857, -0.0006876443, -0.009799323, 0.01795707, 0.04744276, -0.025575982, -0.015901972, 0.035112154, -0.0047837486, -0.0041446625, 0.0034742486, 0.014736578, 0.08305616, 0.014536081, -0.007142729, 0.013095004, 0.027668677, -0.031528257, -0.015225291, 0.013771683, 0.0123619335, -0.016904458, -0.0013737222, -0.018784124, 0.011848158, 0.005939743, 0.06345751, -0.009266751, -0.0069860904, -0.025250174, -0.0022039078, 0.011572475, -0.039873973, -0.037643436, -0.004658438, -0.0063094106, -0.00074755866, -0.007424679, 0.00297927, 0.02576395, -0.00043858864, 0.012819319, -0.026390504, 0.014460893, 0.022079805, 0.0404504, 0.008821897, 0.0071803224, 0.0022477668, 0.022718892, 0.009699075, 0.02031292, 0.037292566, 0.010137663, -0.024999551, -0.024222624, -0.014335583, 0.033583358, 0.035738707, -0.017505951, -0.0115975365, 0.024723867, -0.0006422191, 0.04473604, -0.028220046, 0.034234975, -0.005801901, 0.037944183, 0.005914681, 0.0023746442, -0.00919783, 0.009667747, -0.018708939, 0.0037280035, -0.028796477, -0.009216627, -0.007894595, -0.0051659476, 0.01152235, 0.0131075345, 0.019235244, -0.02520005, 0.036640946, 0.010137663, 0.03403448, -0.033107176, -0.023119887, 0.020388106, -0.018646283, 0.0037092068, -0.0016744687, -0.031829003, -0.007950986, -0.0021365532, -0.013583717, -0.0032612197, -0.025864199, -0.040024344, -0.007174057, 0.0054760925, -0.0055857394, -0.01713002, 0.02053848, -0.005090761, -0.018708939, 0.0051659476, 0.04200426, -0.025814073, 0.013922056, 0.024272747, -0.026415566, -0.023232667, -0.0006735468, -0.01395965, -0.0663146, 0.020651259, -0.006829452, -0.002479592, -0.041377705, 0.04097671, -0.017330516, -0.020012172, 0.01805732, -0.004658438, -0.0030998818, 0.013345625, -0.014059898, -0.043658365, -0.018784124, -0.015012262, -0.04987379, 0.007687832, 0.02656594, 0.0032643524, -0.0032486888, -0.0014927677, -0.020337982, 0.012725336, -0.009160236, -0.0016697696, -0.025037145, 0.0017700184, -0.008452229, -0.024485776, 0.012148906, -0.032756306, 0.016779147, -0.057392456, -0.006910904, -0.015463382, 0.002487424, -0.00043114828, -0.00732443, -0.037944183, 0.0016650704, 0.045262348, -0.015964625, 0.02724262, -0.020074828, 0.0123494025, 0.006948497, 0.04756807, -0.016353091, -0.033984352, -0.010876998, 0.008671524, -0.002405972, 0.033934228, 0.008759242, -0.024272747, -0.017443297, -0.0050500347, 0.05894631, 0.01600222, 0.0049967775, -0.0060337265, -0.03313224, -0.003565099, 0.014736578, 0.039648414, -0.009999821, 0.039222356, 0.001909427, -0.023658725, -0.0029667388, 0.013019817, 0.003258087, 0.016390683, -0.021027192, 0.00925422, 0.021478312, 0.026039634, -0.04531247, 0.008539948, 0.024723867, -0.0038282522, -0.0053069224, 0.014498487, -0.01441077, -0.027167434, -0.027894238, 0.03350817, 0.010563721, -0.0073494925, 0.0046365084, 0.014498487, 0.020175077, 0.000108766064, -0.02452337, -0.0035870285, 0.0009727269, 0.012098781, 0.021315407, 0.016854335, 0.0036841445, 0.015726535, -0.051377527, -0.005341383, 0.037092067, -0.0056577935, 0.03689157, -0.034761284, 0.04107696, -0.014786703, -0.005485491, 0.010870732, -0.015137574, -0.027292743, 0.017505951, -0.0155636305, 0.005748644, -0.02328279, 0.002470194, 0.005927212, 0.039347667, 0.019573584, 0.040249906, -0.0075311936, -0.017994665, 0.02451084, 0.041929074, -0.018195163, -0.01825782, 0.008928412, -0.011058699, -0.028996974, 0.006979825, 0.01908487, -0.030575894, -0.012994755, 0.023207605, 0.030600956, -0.00058857026, 0.0037969244, 0.0899232, 0.005946009, 0.03135282, 0.0071239327, -0.01795707, 0.016503464, 0.016716493, -0.031603444, 0.013383219, -0.025400547, 0.01953599, -0.017405704, 0.012982223, -0.018997153, 0.0014426433, -0.02624013, 0.029423032, 0.005435366, -0.011760441, 0.009204096, 0.01107123, -0.012881975, 0.011979735, -0.0076753013, 0.031027013, -0.026215069, -0.022342958, 0.014698985, 0.009298079, 0.012092515, -0.008709117, 0.005024973, -0.01043841, 0.0074309446, -0.0068545137, 0.010018618, -0.02862104, -0.004542525, 0.0010330329, -0.003815721, -0.01759367, 0.0013165491, -0.02816992, 0.002503088, -0.00258454, -0.043232307, 0.0059209466, 0.011710317, 0.047944, 0.012725336, -0.0049842466, 0.024197562, 0.015613755, 0.0070675425, 0.014385707, -0.021064786, 0.019886862, -0.028395481, -0.012587494, 0.006754265, 0.0352124, -0.0059836023, -0.0076940977, 0.010952185, -0.05654034, 0.005761175, 0.00047226597, -0.006168436, -0.021165034, -0.039172232, -0.006879576, 0.037167255, 0.0024012728, -0.007819409, 0.047893878, 0.021841714, 0.012625087, -0.010062477, -0.0035463024, -0.033057053, -0.000601493, -0.003358336, -0.021077316, -0.010388285, 0.0031672365, -0.0036120906, 0.013746621, 0.009918368, 0.017744044, -0.016904458, -0.01543832, -0.008834428, 0.0010283337, -0.00635327, -0.011935877, -0.005551279, 0.012662681, 0.01939815, 0.008195342, 0.0043138326, -0.0132955015, 0.013896994, 0.055237107, 0.018997153, 0.015262884, 0.03984891, -0.01294463, -0.026916811, -0.0065600327, -0.01713002, 0.010927122, 0.006547502, 0.0062279585, -0.0062436224, 0.012224092, -0.011002309, 0.00479628, -0.0041634594, -0.024573494, -0.016390683, 0.004432878, -0.0036528169, 0.0018906303, -0.008984801, -0.0049529187, 0.020801632, -0.016064875, -0.023232667, 0.015739067, 0.016616244, -0.031027013, 0.013044879, 0.044009235, -0.0011082195, -0.021290345, 0.0131075345, 0.011772972, 0.00976173, -0.013182721, -0.0013463105, 0.04395911, -0.0023104222, -0.020112421, -0.023758972, 0.024322873, -0.001017369, -0.003843916, -0.010620111, 0.02487424, 0.02872129, 0.0342851, -0.011948408, 0.024535902, 0.006321942, 0.00060815015, -0.032981865, -0.026089758, -0.00080277387, 0.045011725, -0.009930899, -0.04393405, 0.010018618, -0.02225524, -0.02634038, 0.008414636, -0.02362113, 0.0028132328, -0.04140277, -0.02249333, 0.008220404, -0.0013494432, -0.0022117398, 0.01746836, 0.0046678362, 0.017581139, -0.030851578, 0.0075750523, 0.0037154723, -0.00987451, 0.004013086, -0.006747999, -0.02598951, 0.0383953, 0.019485867, -0.0038846422, -0.0008294024, -0.0039504305, -0.0022869264, 0.010682766, 0.020062298, 0.005811299, 0.017869353, 0.0043451604, -0.03974866, -0.009724136, 0.008370778, -0.01997458, 0.016102469, 0.02021267, -0.029798964, -0.038696047, 0.0098995725, 0.0004009953, -0.027568428, 0.0027678076, -0.0070550116, 0.033808917, -0.040149655, -0.0018373731, 0.013884463, -0.0008607302, 0.018596157, 0.057943825, 0.004893396, 0.010808077, -0.028420543, -0.008164014, 0.010764218, 0.020688852, 0.01384687, -0.010826874, 0.033432987, 0.0030137305, 0.013220315, -0.0329568, -0.012894506, 0.01646587, -0.00016535183, 0.009122644, -0.0024811586, -0.00084819907, 0.05117703, 0.034811407, -0.032906678, 0.02634038, 0.030926764, -0.0068169204, 0.0036559496, 0.016277904, -0.0010902061, -0.013947119, -0.017756574, 0.018307943, -0.0052505326, 0.04486135, 0.022718892, -0.008721649, -0.015538569, 0.0008568142, 0.015375664, -0.044660855, 0.02032545, 0.014297989, -0.017405704, -0.026365442, 0.027217558, -0.029773902, 0.010444675, 0.028671166, 0.025939384, -0.035337713, 0.01726786, -0.02441059, -0.029022036, 0.001485719, 0.010826874, -0.005535615, -0.014824295, 0.011315587, 0.005006176, 0.022530925, 0.013245377, 0.04576359, 0.004755554, 0.008471026, 0.00635327, -0.007894595, -0.021378063, 0.015112511, 0.020475823, 0.008477292, 0.009298079, -0.014974669, -0.013408281, -0.00840837, 0.03157838, -0.008251732, 0.007242978, -0.01532554, -0.0047743507, -0.016741555, 0.05004923, -0.041628327, -0.008038703, 0.022781547, -0.00035948603], "content_hash": "f0aa652e4730ee839fcb33adc5773a5e"}, {"content": "generation controls - `head_categories_explained.md` - Previous-Token, Positional, BoW, Syntactic, Other - `model_selector_guide.md` - Choosing models, auto-detection, generation settings ### Category 3: Model-Specific Documentation - `gpt2_overview.md` - GPT-2 architecture, why it's a good starter, variants - `llama_overview.md` - LLaMA/Qwen/Mistral architecture, RoPE, GQA differences - `opt_overview.md` - OPT architecture, comparison with GPT-2\n\n### Category 4: Guided Experiments (Step-by-Step)\n- `experiment_first_analysis.md` - Your first analysis with GPT-2\n- `experiment_exploring_attention.md` - Reading attention patterns and head categories\n- `experiment_first_ablation.md` - Removing a head and observing the effect\n- `experiment_token_attribution.md` - Measuring token influence with gradients\n- `experiment_comparing_heads.md` - Systematic comparison across head categories\n- `experiment_beam_search.md` - Exploring alternative generation paths\n\n### Category 5: Interpretation, Troubleshooting, and Research\n- `interpreting_ablation_results.md` - How to read ablation probability changes\n- `interpreting_attribution_scores.md` - Understanding attribution score values\n- `interpreting_attention_maps.md` - Reading BertViz patterns visually\n- `troubleshooting_and_faq.md` - Common issues and frequently asked questions\n- `recommended_starting_points.md` - Best models, prompts, and experiment order\n- `mechanistic_interpretability_intro.md` - Mech interp research context\n\n## Notes\n\n- Large files will be automatically chunked (~500 tokens per chunk)\n- Embeddings are cached in `embeddings_cache.json` for faster subsequent loads\n- Delete `embeddings_cache.json` to force re-indexing of all documents", "source_file": "README.md", "chunk_index": 1, "embedding": [0.0010399428, 0.036995824, 0.030154306, -0.03294255, -0.0399906, -0.027030446, -0.0046632034, 0.020279288, 0.00336267, 0.03100627, 0.03588569, -0.008803612, -0.0051343646, -0.030154306, 0.005883059, 0.022409193, -0.012004926, -0.020382557, 0.0042824023, 0.07099687, 0.04053276, -0.017839577, 0.031393524, 0.051195197, -0.0021847673, -0.012779437, 0.0057926993, 0.032193854, 0.0036531116, -0.02759842, 0.016187286, -0.022435011, -0.040016416, -0.015077153, -0.04776153, 0.0003350165, 0.030928819, 0.015399867, 0.010933518, 0.018523728, 0.014960977, -0.059327565, -0.04306283, 0.034336668, -0.021531414, -0.0049633267, 0.031625878, -0.036118045, -0.014302642, 0.01922079, 0.0041694525, 0.0070545073, -0.021286152, -0.02129906, 0.009081145, 0.010236458, -0.046444863, -0.03477556, -0.0044373046, -0.011094875, -0.00445344, -0.014044472, 0.026927177, -0.0032965138, 0.02811476, -0.05478377, -0.058398154, 0.0319615, -0.027701689, -0.03232294, 0.06061842, -0.0073320405, -0.029276527, -0.017413596, 0.008151731, -0.009307045, -0.0019717768, 0.054525595, -0.018317193, -0.048871662, 0.00486974, 0.021208702, -0.012553537, -0.030076856, -0.035498437, -0.011417587, -0.01148213, -0.023596779, -0.017413596, -0.032090586, -0.032787647, -0.021118341, -0.053854354, 0.004524437, 0.006622072, -0.043682437, 0.0041178185, -0.04427623, 0.0036660202, 0.0018394644, 0.008822975, -0.027056262, -0.012985973, 0.008932698, 0.060979858, -0.029508881, -0.0118306605, -0.013695942, -0.036711838, -0.049052384, -0.08968841, 0.010920609, 0.020821447, 0.015451501, 0.025158709, -0.009707209, -0.024242204, 0.02777914, 0.029560516, -0.022034846, -0.006089595, -0.028269663, -0.006628526, -0.012017834, 0.0017119928, -0.010468812, 0.019788764, -0.032710195, -0.009081145, -0.026165575, 0.008706798, 0.024823088, 0.03400105, -0.044586036, -0.03431085, -0.01589039, -0.027959859, 0.029741235, -0.045876887, -0.012134011, 0.0053344467, -0.028708553, -0.009823386, 0.02777914, -0.0524086, -0.009139233, 0.0019217562, 0.05607462, -0.014328459, 0.026488287, -0.024332564, -0.009223139, 0.0021621773, -0.0077515673, -0.015348232, -0.05215043, -0.009700755, 0.06190927, 0.0066672517, -0.062167443, 0.011850023, 0.030489929, -0.01210174, 0.0007745113, -0.03614386, 0.040584393, -0.034388304, -0.018885167, -0.02244792, -0.048871662, 0.013424863, -0.045360547, 0.00068576523, -0.040016416, -0.0041210456, 0.0059250114, -0.06335503, -0.007441763, -0.012701985, -0.04053276, -0.00048003567, -0.008616438, -0.06227071, 0.03221967, -0.042907927, -0.03299418, 0.016226012, -0.0090424195, -0.030231759, 0.03624713, -0.0049342825, 0.012927885, 0.06077332, -0.007441763, 0.01004283, -0.025468513, 0.0274177, -0.011165871, 0.026643189, -0.0042662667, 0.0074224, 0.0028124442, 0.010752799, -0.016290555, -0.001057692, 0.011391771, -0.02253828, 0.027546786, -0.045980155, -0.06521385, 0.011972654, -0.04585107, 0.026823908, 0.015516044, -0.0733204, -0.010868976, -0.04621251, -0.014031563, 0.026694823, 0.038725566, 0.012727803, -0.0038467397, 0.03826086, 0.04112655, 0.017090883, 0.021002166, 0.018781899, -0.047451727, 0.028243845, 0.004133954, 0.03885465, 0.050265785, 0.013515223, -0.003151293, -0.01685853, -0.03361379, 0.018291375, 0.00017083623, 0.020098569, -0.04401806, 0.022202658, 0.0046470677, -0.019311149, -0.00046107627, 0.024164753, -0.0033368529, -0.02994777, -0.09159887, 0.045463815, 0.033820327, 0.06939621, 0.010204187, 0.0023332154, 0.031290255, 0.01651, 0.00053731725, -0.017852485, -0.00069585, -0.03309745, 0.03815759, -0.027908225, 0.021892853, -0.019943666, -0.005069822, -0.017400688, 0.012437361, -0.09010148, 0.017968662, -0.03988733, -0.06774392, 0.0020250243, -0.016368005, 0.01981458, 0.012889159, -0.046135057, -0.004479257, -0.026088122, 0.016742352, -0.027030446, -0.020240562, 0.019853307, 0.011152963, 0.04443113, 0.020111477, -0.00097620697, 0.034104314, 0.0019766174, 0.01370885, 0.012159828, -0.01887226, 0.018343009, -0.0144833615, 0.030954635, -0.015386959, 0.0073255864, 0.04383734, 0.039422628, 0.039448444, -0.02428093, 0.01345068, 0.027572602, -0.050782125, 0.02235756, -0.017168334, 0.03903537, 0.015761305, -0.025313612, -0.009784659, 0.009119871, 0.014199374, -0.021027982, -0.05065304, -0.024835996, -0.0365053, -0.02208648, 0.01178548, -0.0050181877, -0.012379273, 0.027004628, -0.021273244, -0.036479484, -0.03387196, 0.022809358, 0.012934339, 0.030851368, 0.016884347, -0.029224893, 0.037589617, 0.00020633466, -0.028631102, 0.0455929, -0.012766528, -0.0025591145, 0.04391479, -0.010978698, -0.005066595, -0.0045341183, 0.06128966, -0.012024288, 0.034594838, -0.0937675, 0.016458366, -0.023545144, -0.0023170796, -0.028450383, 0.032090586, 0.008087189, 0.035756607, 0.03580824, -0.0038951465, -0.02182831, 0.0029673465, -0.018601181, -0.017775035, -0.0009858884, 0.016832713, -0.022099389, -0.04569617, -0.0068802424, -0.006622072, -0.0008519624, 0.028863454, 0.041358903, 0.008455082, 0.033820327, -0.027314432, -0.04445695, 0.029767051, 0.0063477657, 0.0023412833, -0.023635503, 0.009197322, -0.008648709, -0.042494856, -0.08163349, 0.01660036, -0.017232876, 0.030076856, -0.02270609, 0.065833464, -0.0026494742, 0.020188928, -0.03493046, -0.037357263, 0.035395168, 0.012979519, -0.02261573, 0.00895206, -0.026359202, 0.007170684, -0.017955754, -0.022744816, -1.1553379e-05, -0.0027640373, 0.032813463, 0.052770037, -0.041694526, -0.0009810476, 0.038364127, 0.029121626, 0.022280108, -0.0127148945, -0.018007388, -0.005605526, 0.021260336, 0.024474557, 0.033691242, 0.05318311, 0.008719707, -0.007441763, 0.02428093, -0.0018265558, 0.016613267, -0.018665724, -0.025610508, -0.009952471, 0.030412478, 0.036195494, 0.024397107, 0.031032087, -0.027366066, -0.06903478, -0.024758546, -0.041358903, -0.047477543, 0.001187584, 0.02628175, -0.01764595, 0.0106882565, -0.025752502, -0.030515745, -0.0022993304, 0.0363504, 0.018962618, 0.011353045, -0.0342334, 0.05659096, 0.020834355, 0.021505598, -0.025778318, 0.037099093, -0.0177234, 0.008455082, -0.016393824, -0.03789942, -0.0067898827, -0.039345175, -0.067124315, 0.016897256, -0.05999881, -0.007912924, 0.021002166, 0.009100508, -0.05622952, -0.024293838, -0.011120692, 0.036556933, -0.018523728, 0.0658851, -0.010655985, 0.023119163, 0.011682212, -0.014160649, -0.0037789699, 0.025094166, -0.01598075, -0.02987032, -0.015541861, -0.004033913, -0.062735416, 0.053338014, -0.021118341, -0.063509926, -0.05509357, -0.042314135, -0.025791226, -0.03387196, -0.029560516, 0.019569319, -0.045644533, 0.00086083706, -0.0073836744, 0.025391063, 0.052356966, -0.012411544, 0.068156995, -0.029818686, 0.0019362783, 0.027624236, 0.001495775, -0.03911282, -0.006289677, 0.021802494, -0.014070289, -0.008061372, -0.005034324, -0.03415595, -0.05669423, -0.0064671696, 0.008674527, 0.03397523, 0.0032206762, -0.004827787, 0.03064483, -0.012024288, -0.020808537, -0.0007196501, 0.028631102, 0.015425684, 0.025042532, -0.04750336, -0.03800269, -0.026591554, 0.021725042, 0.0066930684, 0.041720342, 0.011049694, 0.03152261, 0.007170684, 0.028605284, -0.006628526, -0.03178078, 0.009900836, -0.019672588, -0.026255934, -0.022099389, -0.007900015, -0.07094523, -0.020395465, 0.00079064694, 0.009539397, -0.028140578, -0.0106882565, -0.024048576, -0.053544547, 0.018549547, -0.0050149607, 0.005608753, -0.03624713, -0.013902478, -0.0025929993, 0.0216605, 0.01152731, -0.011249777, -0.00068011775, 0.0009883087, 0.0114434045, -0.012366364, 0.013192509, 0.027882406, 0.014909343, -0.048536044, 0.07347531, -0.03694419, -0.021105433, -0.009842748, -0.00438567, -0.016548725, 0.0296896, 0.00018888798, 0.004304992, -0.0399906, 0.015916208, 0.010849613, -0.016535817, 0.027753321, 0.013089241, -0.002460687, 0.0455929, -0.03064483, 0.023648413, -0.0034530296, 0.01973713, 0.00038866754, 0.023351517, -0.022654455, -0.0059895543, 0.009655574, -0.00090440334, 0.009842748, -0.026178483, 0.0038435124, -0.012805254, 0.038648114, -0.04445695, -7.027833e-07, 0.021789584, -0.003462711, -0.012424452, -0.0039596893, 0.041513808, -0.0011020651, 0.0025284567, -0.011720938, -0.021105433, 0.009849202, 0.016174378, -0.006951239, -0.02759842, -0.0018620543, 0.018898075, -0.0051569543, -0.04386316, 0.043630805, -0.024474557, 0.02375168, -0.047658265, 0.0030609332, -0.0038176954, 0.008306634, -0.0071642296, 0.003254561, -0.027959859, -0.030025221, -0.0074869427, 0.027133713, 0.004911693, -0.012998882, 0.019943666, -0.024216387, 0.04192688, -0.011972654, 0.0064478065, -0.009597486, 0.01240509, 0.029663783, -0.0059217843, -0.04768408, 0.0074094916, 0.02375168, -0.009190868, 0.0037628342, 0.054473963, 0.0075773024, -0.03826086, 0.002397758, -0.0047761532, 0.005434488, 0.011611216, -0.010797978, 0.02837293, -0.00088907446, 0.005834652, 0.00449862, 0.007319132, -0.0060960497, -0.006409081, 0.009326407, -0.01920788, -0.012392181, -0.016187286, -0.035266083, 0.0593792, 0.0181752, 0.010972244, 0.0024897312, 0.025119985, -0.03973243, -0.008526078, 0.01607111, -0.05406089, 0.005492576, -0.015438592, -0.017762126, 0.0015691922, 0.038209226, 0.018898075, 0.025636325, -0.042133417, -0.00019846852, -0.06753739, 0.0064187627, 0.03570497, -0.0059185573, 0.02025347, -0.013837935, -0.005682977, -0.006208999, -0.0017378097, 0.050343234, -0.027572602, -0.0050633675, -0.06061842, -0.03449157, 0.015399867, -0.035575885, 0.020989256, -0.028063126, 0.016535817, 0.02994777, 0.01965968, -0.022060664, -0.021324879, -0.0065672104, -0.00042114052, 0.036815103, -0.012488995, 0.0125858085, -0.0041694525, -0.071048506, -0.03126444, -0.015128788, -0.0050988663, 0.0061702733, -0.029483065, 0.026488287, -0.032193854, 0.044921655, 0.0074030375, -0.051840626, -0.0296896, 0.015386959, -0.0021105434, 0.013489406, -0.020550366, -0.0148577085, -0.0077773845, 0.01754268, 0.017839577, -0.015516044, -0.026049398, 0.008990785, -0.010520445, 0.009332862, -0.012727803, -0.016871437, -0.022654455, -0.0245391, -0.043424267, 0.0064736237, 0.037253994, 0.027753321, 0.015206239, 0.003930645, 0.0023412833, 0.022951351, -0.03869975, 0.015916208, 0.0021476552, 0.008048464, 0.0051020933, -0.011733847, -0.0012964996, 0.03005104, 0.021583049, -0.029018357, -0.023286974, 0.03400105, 0.002723698, -0.015438592, -0.0027753322, 0.01039136, -0.008093643, -0.04035204, 0.04533473, 0.0008398607, 0.028269663, -0.022125207, -0.013579765, 0.018472094, 0.004424396, 0.002397758, -0.011204597, 0.0011415974, -0.023454783, -0.026952993, 0.014573721, 0.021970304, -0.012256642, 0.014263917, -0.002865692, -0.010100919, 0.03869975, -0.016213104, 0.05168572, -0.06877661, 0.056384426, -0.014134832, 0.019349875, 0.013579765, -0.0087907035, -0.0020540685, -0.019943666, -0.014509179, 0.0019653225, 0.006951239, -0.05431906, -0.011624124, -0.029069992, -0.050162517, -0.037331447, -0.025442697, 0.021557232, 0.008893971, -0.032374572, -0.010985152, -0.031574246, -0.10574661, -0.018149382, 0.004898784, -0.013850844, -0.022099389, -0.0035917962, 0.029844502, 0.050265785, 0.00023134492, -0.03658275, 0.021363603, -0.02358387, 0.030412478, -0.026010672, 0.02122161, -0.034749743, 0.0074094916, -0.030877184, 0.04027459, -0.016445456, 0.0040177773, 0.025959037, 0.013044061, -0.0054538506, -0.041100733, -0.014870618, 0.008868154, -0.0079064695, 0.028192213, 0.015490226, -0.025907403, 0.030438295, -0.017620133, 0.031057904, -0.0072610434, 0.040842563, -0.0075256685, 0.003436894, 0.03030921, 0.0034530296, 0.02410021, -0.04042949, 0.005728157, 0.0022719, 0.023067528, 0.011107783, -0.0119532915, -0.00405973, -0.0038144682, 0.013786301, 0.028321298, 0.017323237, 0.0076289363, 0.012043651, 0.056022987, -0.006551075, -0.025610508, -0.03609223, 0.03345889, 0.03562752, 0.012411544, 0.0142381, -0.0065704375, -0.006977056, -0.011036786, 0.00222672, 0.0014667308, 0.0012271163, -0.05258932, -0.0033497615, -0.014380094, 0.002423575, 0.0039241905, -0.009487764, -0.018756082, 0.026978811, 0.03273601, 0.002928621, -0.0006409888, -0.008551896, 0.026514104, -0.010359089, -0.023183705, 0.014689898, 0.017633041, 0.07017072, 0.044844206, -0.035679154, 0.012088831, 0.041849427, 0.03555007, -0.020602, 0.009003694, 0.005692658, -0.051918074, -0.0019153019, -0.00925541, 0.03309745, 0.022912627, -0.0035562979, 0.009849202, 0.01641964, -0.031729147, -0.030360844, -0.028243845, 0.020472916, -0.017671766, -0.008868154, -0.029457247, -0.00089391513, 0.035240266, 0.046186693, 0.0106818015, 0.038312495, -0.014948068, 0.0074611255, 0.047116105, 0.053596184, 0.011159417, -0.016910164, 0.00943613, 0.006499441, -0.012353456, 0.03020594, -0.03660857, 0.034853008, -0.021505598, 0.0076224823, 0.012798799, -0.028321298, 0.010985152, 0.018846443, 0.0016038838, -0.0055054845, 0.011372408, -0.01170803, 0.0048858756, -0.021621775, -0.019014252, 0.001380405, -0.022822266, -0.018046115, 0.01634219, -0.0031367708, -0.024900539, -0.010339726, -0.0049278284, -0.02646247, 0.0035498436, 0.0038725566, 0.0048632855, -0.0033078087, -0.053699452, 0.007712842, 0.031496793, 0.029663783, 0.01405738, -0.027804956, -0.0066414345, 0.016380914, 0.007758022, 0.018730266, 0.025184527, 0.007228772, -0.010204187, 0.009791113, -0.023725864, -0.02175086, 0.0147415325, 0.0074224, -0.015361141, 0.005763655, 0.04714192, 0.049465455, 0.009965379, 0.053492915, 0.018304285, 0.025829952, 0.019078795, 0.006338084, 0.009836294, 0.01676817, -0.0038047868, -0.009010148, 0.0023767815, 0.002004048, -0.008196912, 0.022886809, 0.003311036, 0.03573079, 0.015825847, 0.034646474, -0.024835996, 0.051298466, -0.032245487, -0.031057904, 0.050678857, -0.014341368, -0.013186055, -0.004343718, -0.023041712, 0.03152261, -0.01738778, -0.030438295, -0.008900426, 0.03327817, -0.03449157, -0.007900015, 0.009900836, 0.00072489417, 0.0024623005, 0.012934339, -0.0038499667, 0.012340547, -0.047477543, 0.0012553538, 0.0068092453, 0.0049762353, 0.002768878, -0.0054635317, -0.044095512, -0.021079617, -0.021428147, -0.015877482, -0.010907701, 0.00916505, -0.0014691511, -0.011746755, 0.021453964, 0.015257873, -0.0015191716, -0.010952881, -0.0027140167, 0.0056668413, 0.010417177, 0.007719296, -0.0082808165, 0.009894382, -0.00082695216, 0.0030222076, 0.048794214, 0.014457544, 0.0139799295, -0.028786004, -0.008448628, -0.019324057, 0.03570497, 0.027804956, -0.026410837, -0.016832713, 0.0127148945, 0.025094166, 0.026669007, -0.010501083, -0.0012440588, 0.0019088477, 0.04391479, 0.037150726, 0.035937324, 0.005447396, -0.009642666, 0.010236458, 0.010791524, -0.03712491, 0.014276825, -0.0067963367, -0.016251829, 0.0048342417, 0.036195494, -0.020795628, -0.009778205, 0.042856295, -0.021247428, 0.029018357, -0.03327817, -0.019943666, 0.014780258, -0.025197435, 0.008577713, -0.025481422, -0.02052455, 0.00020714143, -0.010236458, -0.0074611255, -0.0001509692, -0.015232055, -0.036789287, -0.0071642296, 0.0066737058, 0.0022993304, -0.00041912356, 0.007590211, -0.010171915, -0.025274886, -0.0071319584, 0.015064245, -0.0011109397, 0.005311857, -0.0026849725, -0.03415595, -0.007719296, -0.02296426, 0.011998471, -0.04306283, -0.01746523, -0.010313909, 0.019969484, -0.046677217, -0.000964912, 0.013179601, -0.006528485, 0.030593198, -0.037408896, 0.017194152, -0.0047600176, -0.011798389, -0.027236981, -0.0036176133, -0.010623714, -0.030722283, -0.0016111449, 0.014276825, 0.0044631213, 0.0036434303, -0.0033820327, -0.027650055, 0.026049398, -0.013954112, -0.01607111, -0.012088831, 0.007829019, -0.017917028, -0.005518393, 0.022731908, -0.04386316, 0.017439414, -0.038028505, -0.018136473, 0.014728623, -0.016626175, -0.010017013, 0.010914155, -0.011959746, -0.0027333796, 0.034026865, 0.013670125, -0.0025107076, 0.0015595108, 0.017710492, 0.012701985, 0.066607974, -0.023325698, -0.052434415, 0.0005110968, -0.001198879, -0.00038806244, 0.0075837565, -0.01210174, -0.028837638, -0.006534939, -0.016019477, 0.032865096, 0.0077838385, 0.007880652, -0.0027850135, -0.01537405, -0.008564805, 0.004221087, 0.051556636, -0.013424863, 0.03503373, 0.03931936, -0.029044174, -0.022099389, 0.00491492, -0.009674937, 0.030180125, -0.010384906, 0.0023913037, 0.015825847, 0.0006268701, -0.05189226, 0.010862521, 0.02481018, -0.012308275, 0.014534996, 0.01192102, 0.02341606, -0.027030446, -0.0010859295, 0.0076999334, 0.03405268, 0.013928295, -0.016019477, 0.02104089, 0.009971833, -0.0025042533, 9.444399e-05, -0.0066930684, -0.020576185, -0.0055797086, 0.013231235, -0.018084839, -0.028863454, 0.04270139, -0.048768397, -0.009319953, 0.041074917, 0.010068648, 0.025752502, -0.019453142, 0.044250414, -0.029637966, -0.018988436, 0.01634219, -0.06604, -0.024061484, 0.005311857, 0.0171167, 0.026669007, -0.017297419, -0.014664081, -0.012650351, 0.03038666, 0.027727505, 0.008080735, -0.022951351, -0.020576185, 0.015709672, 0.038131773, 0.0028479428, -0.022254292, -0.0108173415, 0.002012116, -0.028631102, -0.002152496, 0.039500076, -0.016690718, -0.008616438, 0.034904644, 0.021079617, 0.010584988, 0.01405738, 0.07254589, -0.01301179, -0.009816931, 0.007758022, -0.023712955, -0.0052795853, 0.0018346236, -0.0029673465, -0.0055280747, -0.03232294, 0.015348232, -0.019117521, -0.013031153, -0.0019959803, -0.005034324, -0.02175086, 0.002171859, -0.005970191, 0.009474855, -0.00041952697, 0.010184824, -0.013282869, 0.019904941, -0.00021198213, 0.050188333, -0.01798157, -0.01790412, 0.028347114, -0.007900015, -0.0022331744, 0.012940793, 0.00014108612, -0.016213104, 0.01781376, 0.0072352267, -0.012540629, -0.033484705, -0.021428147, 0.008616438, -0.011843569, -0.006654343, 0.015180422, -0.0039951876, 0.008267908, -0.00027208743, -0.057520375, 0.028527834, -0.0010601124, 0.028347114, -0.014431727, -0.007312678, 0.004004869, 0.005092412, 0.0041694525, 0.0059927814, -0.011301411, -0.036995824, -0.023041712, -0.0022654457, 0.04115237, 0.013760485, -0.011488585, -0.025274886, 0.0050310963, -0.025984855, 0.008868154, -0.006502668, 0.020421281, -0.0053376737, -0.027262798, 0.004056503, 0.0133474115, -0.0017603997, -0.027908225, 0.035266083, 0.03598896, 0.029379796, 0.0019766174, -0.015464409, -0.023454783, 0.024319656, 0.007880652, -0.006015371, 0.005440942, 0.02479727, 0.005366718, 0.013825027, -0.0042953105, 0.031135354, -0.019904941, -0.022577005, -0.014212282, 0.0024025987, -0.008751978, -0.010823796, -0.002171859, 0.0193886, 0.0013481338, -0.03273601, -0.010881884, -0.0070867785, 0.048252054, 0.01240509, 0.00964912, 0.016032385, 0.0070932326, -0.029483065, -0.005760428, -0.027495151, -0.02479727, -0.008242091, -0.03126444, 0.014689898, -0.0061928635, -0.008984331, -0.015670946, 0.018652814, -0.021970304, -0.022435011, -0.011495039, -0.011288502, 0.010346181, -0.029302346, -0.0035304807, -0.026075214, 0.0051311376, -0.023506418, -0.003611159, 0.0022848083, 0.046806302, -0.019156247, -0.012037197, 0.03588569, -0.025365246, -0.032529477, 0.022667365, 0.0053860806, 0.020395465, -0.045541264, 0.0034820738, 0.01623892, 0.005511939, 0.0042372225, -0.016690718, 0.013618491, 0.020795628, 0.0016974707, -0.014096106, 0.009965379, -0.025158709, 0.047813166, 0.003046411, 0.044870023, 0.006002463, -0.00023275679, -0.0067124316, -0.034543205, 0.025016716, 0.029250711, -0.026772274, -0.044482768, -0.035575885, -0.011972654, -0.020382557, 0.000538124, -0.027185347, 0.009674937, -0.036195494, -0.008680981, -0.003659566, 0.00013110218, -0.0036724745, 0.022551188, 0.025868678, -0.00043808296, -0.044611853, 0.02777914, -0.0036014777, -0.0031077266, -0.009991196, -0.03841576, 0.009720117, 0.044327863, 0.019530594, 0.00077410793, -0.010675347, -0.015412775, 0.021389421, 0.006279996, 0.024745636, 0.0013941204, 0.0052989484, 0.007196501, -0.020950532, 0.00751276, 0.0013570084, -0.009958925, 0.030670648, 0.04146217, -0.008984331, -0.031470977, 0.012075922, 0.0059508285, 0.0069383304, -0.009081145, -0.015451501, 0.040765114, -0.04112655, -0.018265558, -0.0019040069, -0.021544322, 0.03320072, 0.037073273, -0.00711905, 0.03658275, 0.0049859164, -0.023338608, 0.0036014777, -0.0020088886, 0.0020911805, 0.0012924657, 0.025829952, 0.029405612, -0.013076332, 0.009184414, -0.004595434, 0.028450383, -0.013476497, 0.026669007, 0.018549547, 0.009455493, 0.004350172, 0.008784249, -0.04042949, 0.014664081, 0.017271603, -0.015580586, 0.0009140847, 0.021996122, 0.0318066, -0.015709672, 0.015167513, -0.01301179, 0.0105656255, 0.0032368118, 0.029199077, 0.00027894508, -0.017775035, -0.00982984, 0.027056262, -0.03712491, -0.008248545, 0.061806004, -0.0037112, -0.03363961, -0.003975825, -0.0040726387, 0.012934339, 0.024009852, 0.017052157, -0.018033205, 0.03596314, 0.016251829, 0.009184414, -0.0052602226, 0.01537405, -0.03727981, -0.020989256, 0.0040016416, -0.004053276, -0.011720938, 0.015051337, 0.017271603, 0.002012116, -0.01607111, 5.0600396e-05, 0.013528131, -0.01231473, 0.015993658, 0.017103791, -0.004021005, 0.0066672517, -0.010746345, -0.008622892, -0.038648114, 0.029560516, 0.012760074, 0.035266083, -0.028786004, -0.0019023934, 0.0038951465, 0.031832416, -0.02358387, -0.01048172, 0.014470453, -0.012224371], "content_hash": "b44c28180d2b40e1fa856715fd1335e1"}, {"content": "# Recommended Starting Points\n\n## Best First Model\n\n**GPT-2 (124M)** is the ideal starting model because:\n- It loads quickly and runs fast\n- It has a manageable size (12 layers, 12 heads = 144 heads total)\n- It's the most studied model in mechanistic interpretability research\n- Most educational examples and tutorials reference GPT-2\n\n## Good Starter Prompts\n\n### For Exploring Basic Predictions\n\n| Prompt | What It Tests |\n|--------|--------------|\n| `The cat sat on the` | Simple object prediction (mat, floor, bed) |\n| `The capital of France is` | Factual recall (Paris) |\n| `1 + 1 =` | Basic arithmetic |\n| `Once upon a time` | Creative story continuation |\n\n### For Exploring Attention Patterns\n\n| Prompt | What It Shows |\n|--------|--------------|\n| `The cat sat on the mat because it was` | Pronoun resolution: does \"it\" attend to \"cat\" or \"mat\"? |\n| `Alice gave the book to Bob because she` | Gendered pronoun resolution |\n| `The dogs in the park were` | Subject-verb agreement across a prepositional phrase |\n| `I went to the store and bought` | Sequential event prediction |\n\n### For Ablation Experiments\n\n| Prompt | Why It's Good for Ablation |\n|--------|---------------------------|\n| `The cat sat on the` | Simple enough that ablating one head can change the prediction |\n| `The president of the` | Factual prompts show clear ablation effects on knowledge retrieval |\n| `She picked up the phone and` | Action continuation is sensitive to Previous-Token head ablation |\n\n### For Attribution Experiments\n\n| Prompt | What Attribution Reveals |\n|--------|------------------------|\n| `The capital of France is` | \"France\" should have highest attribution for \"Paris\" |\n| `The doctor told the nurse that she` | Which noun drives the pronoun prediction? |\n| `The large red ball rolled down the` | Do adjectives or nouns matter more? |\n\n## Suggested Experiment Order\n\nIf you're new to the dashboard, follow this path:", "source_file": "recommended_starting_points.md", "chunk_index": 0, "embedding": [-0.00023927572, 0.006972066, 0.036465015, -0.0024710028, 0.011730778, -0.01868901, 0.002898111, 0.043022078, 0.008645915, 0.056827873, 0.010465017, -0.010409682, -0.015023143, -0.05234583, -0.007421654, -0.014594305, -0.017374832, 0.011295024, -0.010001595, 0.06479595, 0.012782122, -0.035275336, -0.018882679, 0.011212023, 0.032342643, -0.021220535, 0.0011974593, 0.044294756, 0.0013167729, -0.0061628087, 0.033310983, -0.022354878, -0.06866932, -0.016226655, -0.020501195, -0.0045823352, 0.0009181962, 0.03134663, -0.010907687, 0.048804466, 0.00254017, -0.029133277, -0.0050837984, -0.012415535, -0.031512633, 0.029963285, 0.0071795685, -0.023807393, -0.036299013, 0.03975738, -0.030405955, 0.038429365, -0.021829208, 0.011620111, -0.0064636865, -0.013446128, -0.025080072, -0.030488957, 0.0005174581, -0.03198297, 0.003427241, 0.016212821, 0.027819099, 0.026726255, 0.02119287, -0.0031280923, -0.046923112, 0.04360308, 0.003786911, -0.00085378415, 0.018979514, -0.020473529, 0.009911678, -0.02044586, -0.021884542, -0.016157487, 0.019629687, 0.035386004, -0.015258312, -0.049966477, -0.012048948, -0.007566905, 0.019851021, -0.027736098, -0.018426176, -0.008964085, -0.0025073155, -0.037156686, -0.0312913, -0.064574614, 0.00053388526, -0.03569034, -0.022465548, 0.046895444, 0.027417928, -0.020141525, 0.03054429, -0.034030322, -0.001505254, 0.027777597, 0.02299122, -0.011627028, -0.029990952, 0.003275073, 0.035469003, -0.0065397704, 0.003183426, 0.009088586, -0.048168126, -0.039785046, -0.056993876, 0.0021511037, -0.009309922, -0.001869247, -0.00736632, -0.0063495603, -0.043852083, -0.0020387068, 0.008147911, -0.049136467, 0.00025289302, 0.013204042, -0.0046203774, 0.0051495074, -0.019325351, -0.034196325, -0.00026456502, -0.054891188, 0.01625432, 0.018315507, -0.0070758173, 0.040670387, 0.030461289, -0.02131737, -0.010548017, -0.0025712952, -0.01772067, 0.011841446, -0.050077144, 0.01729183, -0.0025833996, -0.06534929, -0.019657353, 0.030599624, -0.08012343, -0.027085925, -0.0032266558, 0.029714283, -0.049496137, 0.03992338, -0.002766693, 0.037156686, -0.018910347, -0.028441604, -0.019698855, -0.04188773, -0.024097895, 0.013197126, -0.020293694, -0.072210684, 0.047393452, -0.01128119, -0.013923382, 0.010070763, -0.03569034, 0.054946523, -0.048168126, 0.0011793029, -0.0316233, -0.047365785, 0.025135405, 0.005581803, 0.014677307, -0.02838627, 0.03787603, 0.016641658, -0.094676234, -0.00980101, -0.0004599627, -0.050160144, 0.00047422847, -0.002135541, -0.024457566, -0.014594305, -0.0015441605, 0.0040359134, -0.036852352, 0.023005052, -0.0025661078, -0.023461556, 0.023779726, -0.017513167, 0.07381537, 0.039010372, -0.0029862993, -0.03972971, 0.08228145, 0.027639262, 0.026836922, 0.005253258, -0.0013409815, 0.036714017, -0.005408885, -0.035552006, 0.007103484, 0.0037627025, -0.020515028, 0.017181164, -0.08222611, -0.047393452, 0.019145515, -0.05776855, -0.000602188, 0.028524606, -0.07862941, -0.010748602, -0.054918855, -0.006110933, 0.022507047, 0.051515825, 0.009503591, 0.02523224, 0.027473262, -0.03748869, -0.00018610334, 0.027417928, 0.047919124, -0.04722745, 0.009116254, -0.009593508, 0.02946528, -0.0075115715, -0.006432561, -0.0037350354, -0.01729183, -0.026740087, 0.051488157, 0.002228917, 0.008300079, -0.015728649, 0.033476986, 0.028662939, -0.006252726, -0.015936151, 0.060258575, 0.00081617443, -0.030848626, -0.06656663, 0.047448784, 0.064021274, 0.05505719, -0.023267888, -0.023143386, 0.0076775732, -0.0040566637, -0.046978448, -0.020058524, -0.04155573, -0.03181697, 0.013356211, -0.0115301935, 0.048002124, -0.0023361263, 0.03607768, -0.020653363, -0.03544134, -0.061254583, -0.035469003, -0.034362327, -0.07447938, -0.0020542697, -0.026172915, 0.033034313, 0.016032986, -0.05541686, -0.029880283, -0.0154934805, -0.00630806, -0.05804522, 0.0048762965, -0.032148972, -0.024817236, 0.04238574, 0.020100025, -0.022410214, 0.016461823, 0.00021398641, -0.0026093372, -0.01996169, -0.006764564, -0.007269486, -0.04210907, 0.012166533, -0.04028305, 0.0049731308, 0.012214949, -0.013736631, 0.010146847, 0.011848363, 0.03757169, 0.026006915, -0.021912208, 0.047697786, 0.009856344, 0.042275067, 0.010679435, 0.0066538965, 0.019477518, 0.025536576, 0.024042562, -0.017042829, 0.017789835, 0.004312583, 0.007138068, 0.012014364, -0.0008481643, 0.027431762, -0.034085657, 0.013003457, -0.01684916, -0.05029848, -0.016240489, 0.01848151, 0.041306727, 0.01121894, 0.027791431, 0.017333332, 0.020183025, -0.008998669, -0.014538972, -0.018772012, 0.032287307, -0.019062515, 0.0594839, -0.040338382, -0.026643254, -0.020653363, 0.006806065, -0.018315507, 0.0807321, -0.08554615, -0.004447459, -0.03289598, 0.0148294745, -0.041140724, 0.08526948, 0.010236764, 0.03118063, 0.05265017, 0.010783186, -0.022866718, 0.0074354876, -0.001974727, 0.011308857, 0.019325351, 0.043160412, -0.02495557, -0.0102160135, 0.00078115845, -0.0041223727, -0.015659483, 0.011032188, 0.028801274, 0.008971002, 0.0016824952, 0.0007461425, -0.031125296, -0.0010625831, -0.008037243, 0.030295288, -0.0016652034, 0.0008732375, 0.0044820425, -0.0036382014, -0.04144506, -0.02063953, -0.03972971, 0.008452247, -0.014594305, 0.047116783, 0.009870177, -0.004921255, 0.018315507, -0.024167063, 0.059096564, 0.018260173, 0.01464964, -0.012014364, -0.057989884, 0.015548815, -0.0021096035, -0.023212554, -0.025702577, 0.01608832, -0.023046553, 0.07957009, -0.022520881, 0.032148972, 0.05693854, 0.060977913, -0.011461026, -0.028801274, -0.049883474, -0.009475923, -0.007836658, 0.0075945724, 0.034472995, 0.034804996, 0.01564565, -0.020750197, 0.07420271, -0.027736098, 0.029935619, 0.013176375, -0.012180366, 0.031678636, 0.055638198, 0.013625963, 0.010465017, 0.041583396, -0.013916465, -0.061199248, -0.027888265, -0.051294487, -0.06999733, 0.03552434, 0.03333865, -0.01361213, 0.018938014, -0.033310983, -0.010105346, 0.05389518, 0.054559186, -0.018730512, 0.035026334, -0.039563708, 0.026546419, 0.028081933, 0.022202712, -0.0023395848, 0.044958763, -0.015714817, -0.025038572, 0.02343389, -0.027182758, -0.02857994, -0.048002124, -0.05704921, -0.012360201, -0.02846927, -0.011094439, -0.0021303536, -0.006902899, -0.0053293426, 0.005751263, -0.030571958, 0.056634206, 0.0010565309, 0.08886618, 0.0042157485, 0.026532587, 0.006740356, 0.001610734, -0.00804416, 0.002745943, 0.051598825, 0.0022600424, 0.00072106934, 0.003928704, -0.036797017, 0.022064377, -0.03397499, 0.02703059, -0.032065973, -0.020418195, -0.006335727, -0.00018015927, 0.038069695, 0.029658949, -0.03425166, 0.002531524, 0.032591645, 0.05721521, 0.033725988, -0.01837084, 0.03812503, 0.0034099491, -0.0059518483, 0.0018208298, -0.014704973, -0.013134875, -0.014718806, 0.009939345, -0.016295822, 0.01505081, 0.004163873, 0.030212287, -0.019754188, -0.00020371939, 0.016779993, 0.016351156, -0.027238093, -0.024222396, 0.07132535, -0.007629156, -0.00090695656, 0.04352008, 0.03405799, 0.0039044954, 0.021746207, -0.04280074, -0.030793292, -0.040421385, 0.013888799, 0.0036797016, 0.039619043, -0.016212821, 0.06191859, 0.005478052, 0.03314498, -0.026698587, 0.034749664, -0.03245331, 0.004136206, -0.008182495, -0.025979247, -0.01729183, -0.050353814, -0.029492948, -0.03740569, 0.013058791, -0.018550677, 0.004277999, -0.016143654, -0.042966742, -0.0006406623, -0.009648842, -0.005484969, -0.043381747, -0.009054003, 0.002152833, 0.02694759, 0.011205107, 0.010928437, -0.0037730776, 0.010202181, 0.030710293, -0.016102154, -0.019366851, 0.025495077, 0.042247403, -0.024499066, 0.012899706, -0.021372704, 0.015617982, -0.0013781589, 0.018744346, -0.035552006, 0.022327213, -0.022285713, 0.006591646, -0.0033961157, 0.0007971534, 0.010409682, 0.012927373, -0.0054469267, 0.004513168, -0.018578343, 0.0039874963, -0.023530724, -0.0068925237, -0.0037281187, 0.03082096, -0.005772013, 0.015548815, -0.020196859, -0.0040289965, 0.0014931497, -0.0051495074, -0.0003123337, -0.017803669, -0.012650704, -0.013951049, 0.04163873, -0.017305665, 0.010949188, 0.015258312, -0.003928704, -0.011723861, -0.006989358, 0.006110933, -0.018951846, 0.0045200847, 0.010548017, -0.014774141, 0.00060651096, 0.0098286765, -0.01329396, -0.034638997, 0.009787177, 0.008701249, -0.0033269483, -0.024429899, 0.004997339, 0.0018640594, 0.012602286, -0.04750412, 0.012885872, 0.0060175573, -0.011039105, 0.015382813, -0.032010637, -0.015341313, -0.01680766, -0.011177439, 0.0076844897, -0.011461026, -0.015396646, 0.031318966, -0.022631548, 0.006301143, 0.011751529, -0.01884118, 0.0044889594, -0.0018813512, 0.00096401956, 0.00366241, -0.0066054794, 0.014117051, -0.006211226, -0.012145782, 0.009441339, 0.059373233, 0.014220802, -0.020694863, 0.00201104, 0.013003457, -0.0010262702, 0.017070496, -0.007836658, -0.012104282, 0.0075046546, -0.016932162, -0.010437349, 0.018287841, -0.010153763, -0.008175578, 0.016005319, -0.028441604, -0.025329074, -0.0490258, -0.0119313635, 0.06081191, -0.006121308, 0.012000531, -0.014608139, 0.023461556, -0.026809255, -0.033227984, 0.0027632345, -0.026892256, 0.01187603, -0.022368712, -0.033117317, -0.015507314, 0.02076403, 0.002341314, 0.036714017, 0.04304974, -0.014525138, -0.070108004, -0.03704602, 0.022935884, -0.0027839849, -0.0116685275, -0.02478957, -0.0016876827, -0.0315403, -0.0047794622, 0.01837084, -0.0013556796, 0.016240489, -0.04116839, 0.017374832, 0.015590315, -0.01323171, -0.017734502, 0.005211758, 0.021289703, 0.019643521, -0.0028081934, -0.047172114, -0.001416201, 0.021372704, -0.017028995, 0.0040013297, -0.014843308, 0.04587177, -0.021607872, -0.068281986, -0.041362062, -0.016517157, 0.008791167, 0.025246074, -0.011094439, 0.011793029, -0.030571958, 0.05480819, 0.018896513, -0.035911676, -0.05137749, -0.001232043, 0.011364192, 0.020100025, -0.033725988, 0.0127198715, 0.0045477515, -0.0016461824, -0.01983719, -0.035303004, -0.06081191, 0.021483371, 0.011025271, 0.02184304, 0.000483739, -0.032425642, 0.0068233567, -0.008009576, -0.045816436, 0.012685288, 0.02846927, 0.010548017, 0.0027217343, -0.014027134, -0.013625963, 0.014006384, -0.029990952, 0.017859003, 0.0107416855, -0.027417928, 0.001850226, -0.010112263, 0.0071173175, 0.029520614, 0.012014364, 0.014123968, -0.036243677, 0.04352008, -0.025412075, -0.0560532, 0.0014473263, 0.01293429, -0.010963021, -0.017305665, 0.035718005, -0.004900505, -0.001945331, -0.017111996, -0.018619845, 0.04731045, -0.0068890657, 0.009109337, -0.012823622, 3.1314426e-05, -0.041915398, -0.035828676, 0.061143916, -0.0048278794, -0.018025005, 0.02072253, 0.014511305, 0.01088002, 0.03250864, -0.03189997, 0.050685816, -0.0053051338, 0.03640968, 0.033476986, -0.01772067, -0.004921255, -0.012014364, 0.012789038, -0.018938014, -0.0060763494, 0.0061143916, -0.00459271, -0.0056336788, -0.008389996, -0.030156953, -0.025246074, -0.019076347, -0.0210822, -5.4874545e-05, 0.008244745, -0.0013219605, -0.021773875, -0.027542429, -0.11509443, -0.014151635, 0.0007578145, -0.021137534, -0.009254588, -0.0035137, 0.018730512, 0.022036709, -0.006394519, -0.003984038, -0.0042503322, -0.024609733, 0.0098286765, -0.0034981375, -0.005069965, -0.04802979, 0.020528862, -0.028856607, 0.014746474, -0.04415642, -0.0053293426, 0.0033442401, -0.028040433, -0.001346169, -0.04814046, -0.038899705, -0.028441604, 0.006155892, 0.010056929, 0.013999467, -0.03109763, 0.011813779, -0.0016479115, 0.007037775, -0.025246074, 0.037710026, -0.018855013, -0.0022444797, 0.03289598, -0.014663473, -0.016973661, -0.0108731035, 0.02929928, -0.029852618, 0.060922578, -0.035745673, 0.0137712145, 0.01951902, -0.028123435, 0.014953976, 0.03281298, 0.024941737, -0.007193402, 0.023669058, 0.016987495, 0.0033770946, -0.010776269, -0.030018618, 0.017803669, 0.019062515, -0.0024952113, 0.011350358, 0.028012767, -0.027085925, -0.005139132, 0.01233945, 0.011848363, 0.017250331, -0.008991752, -0.021566372, -0.032148972, 0.018052671, -0.011156689, -0.0015899839, -0.01505081, 0.01972652, 0.024706569, 0.03784836, -0.017471666, 0.020196859, 0.024028728, -0.010561851, -0.017776003, 0.012623037, 0.026864588, 0.029575948, 0.0124570355, -0.022963552, 0.018495342, 0.020736365, 0.027597763, -0.023503058, -0.0007807262, 0.0038491616, -0.03674168, 0.017665334, 0.011675444, 0.023973394, 0.039978713, -0.0060832663, 0.010271348, 0.029105611, -0.037627026, 0.027763763, 0.0004802806, 0.051681824, -0.009897844, -0.006135142, -0.0026698587, -0.0053224256, 0.025605744, 0.026020747, -0.007878158, 0.008618249, -0.022783717, 0.0076844897, 0.07569672, 0.034389995, 0.021123702, 0.0034341577, 0.024402233, -0.012055865, -0.02072253, 0.0072833193, -0.041500393, -0.0058031385, -0.010707102, 0.0124570355, -0.014912475, 0.0145666385, 0.02929928, 0.02938228, -0.02531524, -0.0044716676, -0.008957169, -0.017983504, 0.015853152, -0.01195903, -0.027680764, 0.0061524333, -0.0076429895, 0.026089914, 0.01293429, -0.026726255, -0.023586057, -0.024623567, -0.010050013, -0.03992338, 0.017111996, 0.014621972, 0.026823089, 0.020943865, -0.0051252986, 0.0070965677, -0.028773608, 0.0079749925, 0.013667463, -0.020404361, 0.0021995208, -0.012353284, 0.009282255, 0.014497471, 0.033117317, -0.016060652, 0.05688321, -0.041583396, 0.0045304596, 0.009220004, 0.014386804, -0.009067836, -0.025370575, 0.021690873, 0.042330403, 0.02614525, 0.027473262, 0.034638997, 0.015576482, -0.014220802, -0.01761, 0.023392389, 0.008293162, 0.022410214, 0.013861132, 0.002576483, 0.0244714, -0.046812445, -0.012111198, 0.0028064642, -0.015936151, 0.017471666, 0.0031747802, 0.0041915397, -0.006104016, 0.052124497, -0.0063876025, 0.013280126, 0.038623035, -0.017623834, -0.0010487495, -0.029077943, -0.026850756, 0.0059518483, -0.00946209, -0.037073687, 0.008701249, 0.029492948, 0.018011171, -0.0037350354, 0.0017793294, -0.014414471, -0.042607073, 0.0058792224, -0.008272412, 0.012512369, -0.007732907, 0.009441339, -0.013141791, -0.008230912, -0.020971533, -0.025854746, -0.06966533, -0.03156797, -0.02902261, 0.022590049, -0.015631815, 0.026089914, 0.012069698, -0.0056336788, 0.024761902, 0.0043298746, -0.0048970464, -0.01128119, -0.031401966, 0.00289984, -0.018813513, 0.028690606, 0.011710028, 0.020473529, 0.02152487, 0.010243681, 0.048085123, 0.0015424313, -0.0132524595, -0.0050561316, 0.007110401, -0.027113592, 0.033753656, 0.0042987494, 0.013764298, 6.1980405e-05, -0.012152699, 0.04008938, 0.03793136, -0.03541367, 0.0087704165, -0.015797816, -0.0024243149, 0.02846927, 0.007241819, -0.0008308725, 0.015147644, 0.025453575, -0.008438414, -0.04396275, -0.009524341, -0.01581165, 0.004710295, 0.00493163, 0.0015078477, -0.014483638, -0.016337322, 0.038623035, 0.0031765094, 0.03010162, -0.006602021, -0.034085657, -0.0018882679, -0.017098162, 0.015562648, -0.028012767, 0.005069965, 0.0038975787, -0.0052636336, -0.009434423, 0.014497471, -0.008839584, -0.026089914, -0.03793136, -0.02591008, 0.006252726, 0.00455121, 0.0019505186, -0.026103748, -0.00812716, -0.019712688, -0.0014827745, -0.006799148, -0.013964883, 0.021137534, -0.045816436, 0.00368316, -0.0017897045, 0.009738759, -0.034500662, 0.019117849, 0.01668316, 0.021165201, -0.026892256, 0.016503323, 0.011820695, -0.029160945, 0.013529128, 0.00069210556, 0.008493748, 0.006373769, -0.005187549, -0.00040419656, -0.021068366, -0.020017024, -0.012609203, -0.02857994, -0.008244745, 0.0056302203, -0.0026577543, -0.015216812, -0.0019764563, -0.0081064105, 0.018052671, 0.012885872, -0.005021548, -0.016752327, -0.011834529, -0.009475923, 0.013729714, -0.046010103, 0.014123968, -0.051211488, -0.021607872, 0.030571958, -0.013383877, 0.01291354, -0.014068634, -0.005491886, -0.008735833, 0.025190739, 0.0022496672, 0.017693002, -0.0077951574, 0.007490821, 0.040255383, 0.060977913, -0.01505081, -0.049413137, -0.0045062513, 0.0071657347, 0.015922317, -0.0037142853, 0.0026508376, -0.014884808, -0.020376693, -0.009531258, 0.029492948, -0.0036105344, 0.0071726516, -0.0009648842, -0.012809789, -0.023738226, -0.02478957, 0.0523735, 0.0027494011, 0.03344932, 0.036437348, -0.023060387, 0.004288374, 0.017153496, 0.032480977, 0.026518753, -0.04822346, 0.0140133, 0.032397974, 0.003804203, -0.062748596, -0.0055541364, 0.017388666, -0.026532587, 0.018329341, -0.016032986, 0.01121894, -0.03272998, 0.023599891, -0.008867251, 0.0069513163, 0.031706303, -0.035386004, 0.0007569499, 0.016558658, 0.007290236, -0.000644553, 0.009607341, -0.007214152, -0.008182495, 0.003088321, -0.040864054, -0.021746207, -0.009330672, -0.023572225, -0.0018277465, 0.032868315, 0.013328544, -0.0034134074, -0.04459909, 0.029243944, -0.015825484, 0.003539638, 0.0120627815, -0.04244107, -0.03082096, 0.007933492, 0.00031816968, 0.02626975, -0.033227984, -0.018094173, -0.01680766, -0.0007085328, 0.027251925, 0.0091646705, -0.01299654, 0.007359403, 0.025481243, 0.015299812, 0.012823622, -0.01193828, -0.049523804, -0.0032560518, 0.0033390527, -0.0022617716, 0.034888, 0.002671588, -0.036299013, 0.0061143916, -0.013805798, 0.02135887, 0.015673315, 0.03253631, 0.0070654424, 0.013785047, -0.014801808, 0.027556263, -0.014898642, -0.005087257, 0.01513381, -0.0025955038, -0.036575682, -0.008825751, -0.016752327, 0.01680766, -0.02599308, -0.008597499, -0.00043316037, -0.00023668194, 0.008327746, -0.0055783447, 0.011627028, -0.0006674647, 0.019325351, 6.2845e-05, -0.020943865, 0.05594253, -0.013328544, -0.044931095, -0.0015605878, 0.019117849, 0.004277999, 0.0043298746, -0.00034324286, 0.002135541, 0.020127691, -0.01987869, -0.050132476, 0.010119179, -0.03281298, 0.04119606, 0.00918542, -0.01473264, 0.022922052, 0.026449585, 0.00022911676, -0.017527001, -0.044101086, 0.011364192, 0.012000531, 0.031401966, -0.019809522, -0.018661344, -0.01868901, -0.016212821, 0.024485232, 0.0034721997, -0.023420056, -0.0019245808, -0.0099600945, -0.014538972, 0.05721521, 0.013591379, -0.007276403, -0.028275602, -0.0014369512, -0.021981375, -0.01157861, -0.0048866714, -0.0007707834, -0.0127198715, -0.0152306445, 0.017305665, -0.0009968741, 0.0279851, -0.009807927, 0.004955839, -0.0068994407, 0.0009847698, 0.014884808, 0.0043506245, -0.022797551, -0.012138866, 0.008217079, -0.009088586, -0.0019107474, 0.0035621172, -0.017028995, 0.006895982, -0.016821494, 0.016766159, -0.026933756, -0.025301408, -0.013944132, 0.031208297, -0.019104015, 0.0045270016, 0.007649906, 0.0023637933, -0.029714283, -0.060147904, 0.0050077145, 0.005401968, -0.0025367117, 0.017084328, -0.012387868, -0.020888532, 0.04127906, -0.026366584, -0.011295024, -0.01469114, 0.0004724993, -0.01608832, -0.03272998, 0.041915398, -0.0010764165, -0.009925511, 0.0020542697, -0.009060919, -0.006550146, -0.009628092, -0.012028198, -0.007573822, 0.029880283, -0.00980101, -0.0072971527, -0.0043852082, -0.000759976, 0.009669592, -0.01928385, -0.02000319, 0.03261931, 0.0064706034, -0.011205107, 0.016434157, 0.007234902, -0.037737694, 0.030322956, 0.026574086, 0.0076706563, -0.014511305, 0.0139718, 0.0032819896, 0.012138866, -0.0070965677, -0.015880818, -0.0046203774, 0.021538705, -0.0037938277, -0.0083692465, 0.04352008, -0.007144985, 0.017803669, -0.025674911, 0.05289917, 0.007013567, 0.0055887196, -0.034362327, -0.0038560783, 0.050409146, -0.01121894, -0.035386004, -0.0060936413, -0.006674647, -0.02929928, -0.022977386, -0.0013098562, -0.012214949, 0.013722797, -0.044460755, -0.016641658, 0.006778398, 0.013031124, 0.021414204, 0.02411173, 0.0072556525, -0.02383506, -0.011620111, -0.008396913, -0.00812716, 0.00153465, -0.013003457, -0.01892418, -0.0071657347, 0.025757913, 0.023295555, -0.010527267, 0.0006155892, -0.037903693, 0.0022237294, -0.008728917, -0.0038076611, 0.0043990416, 0.0043091243, 0.014815641, -0.0015666399, 0.01729183, 0.012899706, -0.005118382, 0.001020218, 0.032563977, 0.006318435, 0.0013954508, 0.014234635, 0.001657422, -0.018716678, -0.032038305, -0.050243143, 0.045041762, -0.014787974, -0.019809522, 0.030322956, -0.014220802, 0.028524606, 0.025190739, 0.007525405, 0.030710293, -0.04459909, -0.016295822, 0.0073386533, -0.0032422184, -0.0043817502, -0.018938014, 0.010568767, -0.00031492749, -0.019947857, 0.013736631, -0.028552271, 0.0046307524, -0.009330672, 0.01636499, 0.025744079, 0.028234102, 0.013079541, 0.026338918, 0.0016487761, 0.017706836, 0.00048762964, -0.021787707, 0.028413936, 0.03972971, 0.027141258, 0.030239955, 0.0007431164, -0.013370044, -0.02288055, -0.004305666, 0.033476986, 0.0010003324, -0.02510774, -0.01013993, -0.008452247, -0.021690873, 0.0011723861, 0.046369776, 0.014262303, -0.017194998, -0.0001514116, 0.023337055, 0.0065363124, 0.023682892, -0.0037696192, 0.0022859802, -0.00078331993, 0.0016799014, 0.004731045, 0.019076347, 0.021566372, -0.012540036, -0.037737694, -0.024056396, -0.006550146, -0.0020421653, 0.003456637, 0.024499066, -0.02631125, -0.056136202, -0.015825484, -0.042966742, 0.0041604144, 0.017111996, 0.04127906, -0.012470868, -0.016019152, 0.008860335, -0.043464746, -0.03668635, 0.03486033, 0.0140548, 0.030046286, -0.029243944, 0.0019003722, -0.009814844, 0.03452833, -0.0106102675, -0.020736365, -0.0019401434, -0.0073317364], "content_hash": "89df1e1e3909d26f35a5acae2121dcbf"}, {"content": "highest attribution for \"Paris\" | | `The doctor told the nurse that she` | Which noun drives the pronoun prediction? | | `The large red ball rolled down the` | Do adjectives or nouns matter more? | ## Suggested Experiment Order If you're new to the dashboard, follow this path:\n\n1. **Experiment: Your First Analysis** -- Learn the basics with GPT-2 and a simple prompt\n2. **Experiment: Exploring Attention Patterns** -- Understand what attention heads do\n3. **Experiment: Your First Ablation** -- Remove a head and see what happens\n4. **Experiment: Token Attribution** -- See which input tokens drive predictions\n5. **Experiment: Comparing Heads** -- Systematically compare head categories\n6. **Experiment: Beam Search** -- Explore alternative generation paths\n\n## After the Basics\n\nOnce you've completed the guided experiments:\n- **Compare models**: Run the same prompt on GPT-2 and Qwen2.5-0.5B to see architectural differences\n- **Try longer prompts**: See how attention patterns change with more context\n- **Combine techniques**: Use attribution to find important tokens, then ablate heads to find the components that process those tokens\n- **Explore edge cases**: Try prompts in other languages, code snippets, or mathematical expressions", "source_file": "recommended_starting_points.md", "chunk_index": 1, "embedding": [0.005553829, 0.013928651, 0.028761858, -0.012786455, 0.011046332, -0.020728163, -0.013100751, 0.025404265, 0.009842811, 0.024745012, 0.027611997, -0.028547218, -0.016956618, -0.02193935, -0.033391967, -0.0035166584, -0.01760054, 0.001277304, 0.0004884514, 0.029160477, -0.02629349, -0.0053890157, 0.026615452, 0.047036983, 0.033882573, 0.00022422291, -0.007121473, 0.04099638, -0.0007402231, 0.007562253, 0.01717126, -0.017922502, -0.0314602, 0.010210766, -0.038420692, -0.0054848376, -0.013591358, -0.0071444702, 0.004783422, 0.0057493057, -0.0008010699, -0.030402327, -0.037746105, 0.027182717, -0.03881931, -0.0071061417, 0.0054235114, -0.060682002, -0.020636173, 0.04179362, -0.050563224, 0.060620677, -0.020206893, 0.042560194, -0.030923598, 0.0049520684, -0.031230228, 0.028301915, -0.0034189203, -0.025082303, 8.3604486e-05, -0.026646115, 0.034802463, 0.052127037, -0.009022577, -0.01893438, -0.051728416, 0.00499423, -0.0022978059, -0.023411172, 0.037654117, 0.04068975, -0.005764637, 0.008026031, -0.01947098, -0.038022075, 0.009045574, 0.02328852, -0.02753534, -0.033913236, 0.01065538, -0.024300398, 0.018091148, -0.02431573, -0.015155169, -0.038757984, -0.011651926, -0.04823284, -0.05476405, -0.09260215, -0.008830933, -0.0021368251, -0.0069834897, 0.016910624, 0.025327608, 0.024653021, 0.018888384, -0.013169742, 0.015400473, 0.024177747, 0.048815437, -0.011720917, -0.0212341, 0.0088692615, 0.030279675, -0.013422712, 0.009030242, -0.0060980967, -0.01569177, -0.022644598, -0.07518558, -0.007926376, -0.044246655, -0.0010176271, 0.00030495276, 0.028715864, -0.00090072455, 0.0182138, 0.01583742, -0.027259374, -0.015760763, -0.002012257, 0.010548059, 0.012311179, 0.018612418, -0.039187267, 0.030892935, -0.041180357, 0.015300819, 0.01355303, -0.0134303775, 0.011636594, -0.009528516, -0.03176683, -0.0097278245, 6.774898e-06, 0.024791006, 0.01061705, -0.07941707, 0.03397456, -0.009114565, -0.03894196, -0.02463769, 0.027397357, -0.08125685, -0.031046249, -0.005730141, 0.043296102, -0.02131076, 0.021770703, -0.0008331702, -0.006293573, -0.022077333, 0.01474122, -0.0240091, -0.067642495, -0.007934041, 0.015040183, -0.029022494, -0.05485604, 0.024055094, 0.02276725, -0.021341423, -0.032196112, -0.02701407, 0.043725386, -0.041640304, -0.019854268, -0.039800525, -0.03489445, -0.024561033, -0.02701407, 0.024377055, -0.011651926, 0.03397456, -0.0047757565, -0.062061835, -0.016021399, 0.0011690254, -0.027826639, -0.0027730817, 0.024714347, 0.003351845, -0.012142533, -0.014557241, -0.030708957, -0.033821248, 0.012295848, 0.011046332, -0.0096741645, 0.051605765, 0.011858901, 0.09211154, 0.035936993, -0.004848581, -0.049857978, 0.082054086, 0.030034373, 0.0027270871, 0.0004201784, 0.016190045, 0.021464074, 0.037746105, -0.04044445, -0.0038347868, -0.005699478, 0.0065695397, -0.02328852, -0.059118193, -0.05433477, 0.04035246, -0.05694112, 0.034403846, 0.042989474, -0.08916789, -0.01778452, -0.06135659, 0.037102185, 0.04409334, 0.04688367, 0.006358732, 0.022951227, 0.046086434, 0.005308525, -0.008531969, 0.047680907, 0.060038082, -0.026247496, 0.020513522, -0.026201501, 0.039371245, 0.0064238906, -0.016542668, -0.036182296, -0.039831188, -0.04896875, 0.026278159, -0.008922922, 0.007842053, -0.018857721, 0.011421953, 0.014518913, 0.01967029, 0.011360628, 0.016374022, 0.021249434, -0.04148699, -0.06506681, 0.040199146, 0.032748044, 0.034587823, -0.026784098, -0.040321797, 0.025818214, 0.0013913319, -0.0061517567, -0.012395503, -0.01810648, 0.017753856, -0.001074162, -0.012817118, 0.013323057, -0.020329544, 0.014135626, -0.021142112, -0.027182717, -0.021954682, 0.0072632893, -0.051973723, -0.040045828, -0.019501645, -0.04136434, 0.033913236, 0.022905232, -0.065250784, 0.00114507, 0.0007641785, -0.0017832429, -0.032042798, -0.02794929, -0.038665995, 0.00747793, 0.01770786, -0.0030490484, -0.05010328, 0.007221128, -0.00047335948, 0.011329965, -8.8635126e-05, 0.0051590432, 0.018719738, -0.034219865, 0.038420692, -0.04065909, -0.0018589421, -0.008991914, 0.003712135, -0.010778031, -0.0034016725, 0.010019123, 0.048784774, -0.031552188, 0.05136046, 0.023671808, 0.008907591, 0.041517653, 0.0012840115, 0.026155507, 0.045933116, 0.009559179, 0.004584113, 0.0037907087, 0.013928651, 0.005703311, -0.030019041, -0.009076237, 0.034373183, -0.009482522, 0.027443351, -0.017017944, -0.036151633, -0.052770957, 0.022736587, 0.017845845, -0.005162876, 0.004576447, -0.020651504, 0.028608544, -0.023012554, 0.005404347, 0.011015669, 0.002717505, 0.0066921916, 0.06561874, -0.032625392, -0.03179749, -0.028225258, 0.057891674, 0.027566003, 0.07831321, -0.07978503, -0.015247158, -0.021510068, -0.014296606, -0.01954764, 0.0513298, 0.038022075, 0.03363727, 0.038236715, -0.011299302, -0.032012135, 0.0008010699, -0.016527336, 0.020728163, 0.014909865, 0.06592537, 0.00024602236, -0.019992251, 0.010755034, -0.017646534, -0.021065455, 0.013438043, 0.037316825, 0.018183136, 0.00048150434, 0.0041126697, 0.0066193673, -0.009490187, 0.0019595549, 0.021617388, -0.007121473, -0.008049028, 0.015638111, -0.03434252, -0.057401065, -0.009911803, -0.055101342, 0.025419597, -0.0364276, 0.018796396, 0.005645818, -0.012587146, 0.007297785, -0.04326544, 0.052004386, 0.03682622, 0.01288611, -0.0020563349, -0.05454941, 0.0017928251, -0.008961251, -0.037347488, -0.027903296, 0.02992705, -0.04111903, 0.052862946, 0.0041586645, 0.04182428, 0.040597763, 0.058320954, -0.010916015, -0.026508132, -0.036151633, -0.028010616, 0.048907425, 0.046393063, 0.020835483, 0.01267147, 0.012311179, -0.0034323353, 0.057002448, -0.01423528, 0.050655216, 0.0076734065, -0.0043311436, 0.038114063, 0.03621296, 0.0020620842, 0.03259473, 0.050072618, -0.016006067, -0.045473173, -0.016312696, -0.03817539, -0.04489058, -0.0022268977, 0.036672905, -0.030341001, 0.051299136, -0.01594474, -0.009129897, 0.023441834, 0.04026047, -0.034710474, 0.009965463, -0.03320799, 0.0471903, 0.05249499, -0.013422712, 0.007872716, 0.03176683, -0.016343359, -0.031214897, 0.0009697162, -0.042069584, -0.022997223, -0.025235618, -0.05267897, 0.018305788, -0.036795557, -0.022629267, -0.018581755, -0.00011229112, -0.0028995664, -0.015500127, -0.021755371, 0.062583104, -0.0103180865, 0.08101155, -0.014909865, 0.030570975, 0.023242526, -0.002937895, 0.03198147, 0.010847023, 0.042498868, -0.014534244, -0.026048187, 0.025634237, -0.02535827, 0.015208829, -0.028071942, -0.016833967, -0.023855785, -0.011989218, -0.01247216, -0.013399715, 0.04338809, 0.0025507752, -0.06390161, 0.016895292, -0.007270955, 0.028531887, 0.03930992, -0.045841128, 0.06893034, -0.008593296, -0.004323478, 0.011061664, -0.006611701, -0.013591358, -0.03851268, 0.014388595, -0.020237556, 0.0298044, -0.0052395337, 0.01257948, -0.03912594, -0.017125264, 0.009367535, 0.03238009, -0.030463653, -0.034373183, 0.059148856, 0.017263249, 0.020513522, 0.05182041, 0.02370247, -0.016082725, 0.02557291, -0.00022039004, -0.044675935, -0.01569177, 0.020850815, -0.01293977, 0.0008403568, 0.003907611, 0.053537533, 0.024714347, 0.008777273, 0.0054273442, 0.04252953, -0.011743914, -0.012955102, 0.0034974942, -0.050839193, -0.04998063, -0.055377312, -0.0019241009, -0.035967655, 0.025189623, -0.05102317, -0.0072096293, -0.01873507, -0.061295263, -0.0088156015, 0.018259794, -0.013208071, -0.044154666, 0.0015915994, 0.007926376, 0.031030918, 0.034618486, -0.0033403465, 0.006998821, 0.00068368827, 0.02267526, -0.03986185, 0.00424682, 0.024561033, 0.018689075, -0.039401907, -0.01474122, -0.020590179, 0.009114565, -0.00838632, 0.00988114, -0.028117936, 0.005178208, -0.026278159, 0.039401907, 0.01657333, 0.008754276, -0.020712832, -0.0037677116, 0.019210346, 0.020636173, -0.019685622, 0.006186253, -0.047220964, 0.021602057, 0.012326511, 0.015047849, -0.012219191, 0.023319183, -0.00036556, -0.0055614947, 0.019087695, 0.012334176, -0.015093843, -0.002813327, -0.00957451, -0.0021674882, 0.042100247, -0.020881478, 0.010210766, 0.030371664, -0.017431894, -0.014948194, 0.0032962684, 0.02255261, -0.009114565, 0.033177327, 0.0049788984, -0.017830513, 0.010839357, 0.0347718, -0.031092243, -0.024928989, 0.025020977, 0.0028823186, -0.018443773, -0.028685201, 0.0042161574, -0.0037236335, -0.0036182296, -0.0029168143, 0.010663046, 0.0019058947, -0.0042889817, 0.004737428, -0.04430798, -0.043541405, -0.015507793, -0.01822913, 0.025020977, 0.0048255837, -0.007098476, -0.00039885807, -0.015714768, -0.0027462516, 0.007002654, -0.0069681583, 0.002884235, 0.00082646264, 0.0314602, 0.015852751, -0.009973128, 0.025480922, 0.01965496, -0.042897485, 0.031230228, 0.030616969, 0.017769188, -0.0012265185, 0.0198696, 0.003073962, 0.010548059, 0.024223741, -0.0051897066, 0.0059294505, 0.0069106654, -0.017033275, -0.014948194, 0.0030586305, -0.016849298, 0.0027769145, -0.005162876, -0.005086219, -0.021663383, -0.05019527, -0.02949777, 0.05497869, 0.028945837, 0.009191223, -0.018459104, 0.0041126697, -0.014503581, -0.037531465, 0.016404685, -0.041333675, -0.018673744, -0.026324153, -0.05362952, -0.016404685, 0.009421195, 0.007121473, 0.034495834, 0.028393904, 0.0126254745, -0.052862946, 0.020544184, 0.034311853, 0.011000338, -0.017263249, -0.026247496, -0.004196993, -0.046607703, 0.0031889481, 0.038328703, -0.0046761017, 0.0026676778, -0.053016264, -0.012479826, 0.032840032, -0.010663046, 0.0035933158, 0.0035434887, 0.022123327, -0.012142533, 0.0036009818, -0.036304947, 0.00797237, -0.0056879795, -0.009313875, -0.013706344, -0.0064047263, 0.04841682, 0.0012993431, -0.08640823, -0.03443451, -0.024147082, -0.011636594, -0.02402443, -0.0029474774, -0.014702891, -0.018995706, 0.052648306, 0.029436445, -0.021188106, -0.03716351, -0.01822913, 0.00988114, 0.01967029, -0.025726225, 0.01355303, -0.007980036, 0.005668815, -0.008110354, -0.027351363, -0.03955522, 0.009873474, -0.011437285, 0.02215399, 0.013928651, -0.023625813, 0.0029168143, -0.0073437793, -0.035967655, 0.023825122, 0.0077040694, 0.009551513, -0.014388595, -0.040751077, -0.002175154, 0.017922502, -0.036580913, 0.032042798, -0.017585209, -0.013269397, 0.044553284, -0.026308822, 0.01583742, 0.0070409826, 0.0131620765, 0.027795976, -0.00449979, 0.03434252, -0.029635753, -0.055745266, -0.02836324, 0.015852751, -0.005852793, -0.0060750996, 0.019317666, -0.01976228, 0.0072632893, -0.027719319, -0.013154411, 0.025174292, 0.005078553, 0.01293977, -0.033453293, 0.005825963, -0.01603673, -0.04531986, 0.036979534, 0.012426166, -0.018305788, 0.01822913, 0.019241009, -0.0072326264, 0.040965717, -0.032472078, 0.05175908, -0.016328027, 0.03851268, 0.011989218, -0.039156605, -0.0021943182, -0.008516638, 0.0041624974, -0.043020137, -0.024867663, 0.014212283, -0.018336453, 0.0023265523, -0.036519587, -0.01603673, -0.032318763, -0.023242526, -0.014603236, 0.004254486, 0.00036556, 0.005699478, -0.012771124, -0.0067228545, -0.1258408, -0.01263314, 0.01965496, -0.02701407, 0.0028689033, -0.0026753435, 0.0339439, 0.03556904, -0.0043081464, 0.020068908, 0.033575945, -0.030678295, -0.013637353, -0.011053998, -0.011552271, -0.032993346, 0.013836662, -0.019716285, -0.003857784, -0.02411642, -0.02647747, 0.004810252, -0.0051283804, 0.012242188, -0.06285907, -0.013890322, -0.028041279, 0.010042121, 0.02640081, 0.0018589421, -0.027090728, -0.0028420733, -0.017339906, 0.011053998, 0.016328027, 0.015822088, 0.0017535381, 0.012978099, 0.041027043, -0.03385191, 0.0030260512, -0.023457166, 0.005615155, -0.038267378, 0.0355997, -0.018673744, -3.961631e-05, 0.0051015504, -0.03250274, 0.044951905, -0.0015388974, -0.0017382067, -0.0052165366, 0.0036546418, 0.034035888, -0.003233026, -0.013821331, -0.03507843, 0.0051667094, 0.027611997, 0.0015666857, 0.010034454, 0.014388595, -0.03351462, -0.0027098393, -0.0073744426, 0.007293952, 0.02078949, -0.025833545, -0.0380834, -0.03581434, 0.022000676, 0.010371747, 0.01484854, -0.011260973, -0.013591358, 0.03860467, 0.020099573, -0.019011037, 6.156548e-05, 0.017876508, -0.037010197, -0.034741137, 0.028746527, -0.0075009274, 0.021356754, 0.031644177, -0.020329544, -0.021387417, 0.027657993, 0.036856882, -0.030969592, 0.010203101, 0.0032272767, -0.049796652, 0.00034136503, 0.008823267, 0.026630783, 0.033667933, -0.001352045, -0.00540818, 0.021142112, -0.032625392, -0.011881898, -0.014633899, 0.039463233, -0.016588664, -0.0073476126, -0.020436864, -0.009497853, 0.03354528, 0.040751077, -0.004879244, 0.008608627, -0.020160899, 0.023457166, 0.043449417, 0.018075816, -0.0040053497, -0.004465294, -0.0021655718, 0.008577963, -0.0037753773, 0.013760004, -0.022031339, -0.0019011036, -0.030647632, 0.0036182296, 0.010341085, -0.013928651, 0.028501224, 0.01988493, -0.016450679, -0.0043541407, 0.012165531, -0.0032291932, 0.017677197, -0.020099573, -0.021080786, 0.008838599, -0.021172775, -0.0011076995, 0.020467527, -0.01081636, -0.03912594, -0.019302335, -0.008807936, -0.009451858, 0.0019595549, -0.00057972164, 0.005469506, 0.017631203, 0.018811727, 0.026124844, -0.019854268, 0.0056573167, 0.005707144, -0.023012554, -0.02391711, 0.015676439, 0.0023552987, 0.017615872, 0.015354479, -3.4465887e-05, 0.06598669, -0.016067393, -0.008378655, -0.023257857, 0.005438843, 0.010540393, -0.03737815, 0.0438787, 0.024913657, 0.04338809, 0.025128298, 0.023656476, 0.026370147, -0.012909107, -0.01412796, 0.046730354, 0.01977761, 0.0039440235, -0.010893017, -0.0014651146, 0.040505774, -0.0031774496, -0.0037332156, 0.007742398, -0.0037772937, -0.00538135, -0.025511585, 0.02847056, 0.008317329, 0.027244043, -0.027887965, 0.022751918, 0.026109513, -0.008363323, -0.007098476, -0.025588242, -0.00081400585, 0.008700616, 0.0027558338, -0.02710606, 0.022690592, 0.028010616, -0.024070425, 0.0016653822, 0.0052242023, -0.010977341, -0.031950805, 0.012924438, -0.037531465, 0.024331061, -0.003911444, 0.0031065415, 0.0066385316, 0.019271672, -0.0014229531, -0.03556904, -0.06801045, -0.006653863, -0.03406655, 0.02121877, -0.01153694, 0.008554966, 0.036519587, -0.031828154, 0.0071674674, 0.0071713002, -0.016512005, -0.04274417, -0.0072517907, 0.004664603, -0.04099638, 0.018045153, 0.0049482356, 0.008685284, 0.024499707, 0.010387079, 0.03293202, 0.010057452, -0.0033920903, 0.0020237556, -0.0014670311, -0.031950805, 0.023487829, 0.033115998, 0.015914077, 0.019394323, -0.006389395, 0.023763796, 0.023104543, -0.038757984, 0.02991172, -0.053476207, 0.018704407, 0.028025948, 0.016297365, 0.009919468, 0.008340326, 0.04501323, -0.012740461, -0.03363727, 0.015047849, -0.016956618, -0.00977382, 0.012694467, 0.03011103, -0.026416142, -0.029865725, 0.03464915, 0.012970434, 0.017523883, 0.0015360228, -0.007217295, 0.003277104, -0.008317329, 0.014012974, -0.005576826, 0.008248337, 0.012564149, -0.016220707, 7.923262e-05, -0.01698728, -0.0064047263, -0.037102185, -0.024622358, -0.004572614, -0.0013012595, -0.01625137, 0.0009189307, -0.004369472, 0.00894592, 0.0015963905, 0.0040896726, -0.011552271, 0.009789151, 0.037654117, -0.055653278, 0.015331482, 0.0052433666, 0.036918208, -0.037531465, 0.020574847, 0.01677264, 0.005772303, -0.041548315, 0.0206975, 0.0051973723, -0.024239073, 0.009091568, -0.010893017, 0.025174292, 0.0007315991, 0.0012753876, 0.0071713002, -0.012211525, -0.024269735, -0.002219232, -0.024729678, 0.0066691944, -0.010141775, 0.0035952323, 0.0051897066, -0.021050123, 0.0068378407, -0.010686043, 0.007995367, -0.009022577, 0.0016730479, -0.020099573, -1.2479287e-05, -0.008899925, -0.033453293, 0.030908266, -0.041456327, -0.034925114, 0.017953165, -0.02526628, -0.0030720455, 0.016113387, 0.013537698, -0.022598604, 0.020820152, 0.023257857, 0.009743156, 0.0021157444, 0.011299302, 0.030754952, 0.08401652, -0.022337968, -0.013131414, -0.0002879444, 0.0033020177, 0.036918208, 0.0017238334, -0.016128719, -0.037286162, -0.0061632553, -0.012203859, 0.02607885, 0.00040053495, -0.00015499169, 0.0075469217, 0.016818635, -0.028409235, -0.023472497, 0.026722772, 0.018428441, 0.000103068276, 0.062981725, -0.009237218, 0.001677839, 0.033790585, 0.023978436, 0.013415046, -0.014894534, 0.009835145, 0.010877686, -0.0038367033, -0.023932442, -0.0027060064, 0.011031001, -0.030448321, 0.030908266, 0.0053660185, 0.013001096, -0.043142788, 0.011498611, 0.009275546, 0.012556483, 0.024852332, -0.0264928, 0.008524303, 0.010433073, 0.0061824196, -0.002479867, -0.017876508, -0.0148792025, -0.02690675, 0.014549576, -0.023779128, -0.01822913, 0.0061517567, -0.036672905, 0.011276305, 0.023380509, 0.016481342, 0.018995706, -0.03955522, 0.004817918, -0.03158285, 0.013691013, 0.022353299, -0.043817375, -0.027198048, 0.029375117, -0.012955102, 0.020421533, -0.020191561, -0.008784939, -0.021433411, -0.013652684, -0.00035334274, 0.032318763, -0.009421195, 0.009796817, 0.02236863, 0.013867325, 0.0051705423, -0.012840115, -0.04026047, -0.007270955, -0.0026657614, -0.02111145, 0.044430632, -0.005358353, -0.040843066, 0.02278258, -0.040383123, 0.014564907, 0.031521525, 0.043786712, 0.017155927, 0.0026523462, -0.022521947, 0.017309243, 0.010065118, -0.010916015, 0.012932104, -0.00085904205, -0.04348008, 0.015929408, 0.007052481, 0.010440739, -0.027581334, -0.004584113, 0.001479488, 0.0048255837, 0.020712832, 0.0052165366, 0.00045419513, 0.0037466309, 0.002589104, 7.551952e-05, -0.012479826, 0.029957714, -0.027918627, -0.04338809, 0.0017937833, 0.00024554326, -0.0038424525, 0.0017104184, 0.00015690814, 0.011452616, 0.045411848, -0.007152136, -0.009352203, 0.019440318, 0.0001244485, 0.037531465, 0.010724371, -0.012219191, 0.021203438, 0.035936993, 0.0028075776, -0.010494399, -0.030524978, 0.019057032, -0.004768091, 0.026676778, -0.011828238, -0.023058549, 0.010111112, -0.03443451, 0.010042121, -0.006776515, 0.0033633437, -0.0054465085, -0.0029225636, -0.018551093, 0.010877686, -0.009382866, 0.001818697, -0.032472078, -0.019808274, -0.032318763, -0.016680652, -0.0076044146, 0.015392807, -0.015078512, -0.020574847, 0.006412392, 0.005626654, 0.035385057, -0.027121391, -0.0018397778, 0.012234522, -0.015354479, 0.012311179, -0.019900262, -0.044277318, 0.013238735, -0.0071406374, -0.015929408, -0.0034246696, 0.0035089927, 0.0063740634, 0.018566424, 0.000120376084, 0.0006645239, -0.019195015, -0.026308822, -0.023871116, 0.030892935, -0.032441415, 0.010241429, -0.019700954, -0.002259477, -0.03271738, -0.04832483, 0.010640048, -0.0010952427, 0.02836324, 0.03633561, -0.026952744, 0.013737007, 0.031230228, -0.013077754, -0.008761941, -0.011414288, -0.014012974, -0.023012554, -0.027918627, 0.025818214, -0.00665003, -0.014764217, -0.0027807474, 0.0033230986, -0.017738523, -0.015178166, -0.020850815, 0.013936317, 0.009850477, -0.006385562, -0.0007224961, 0.007592916, -0.006389395, -0.016603995, 0.0119815525, -0.01925634, 0.02131076, 0.0043656393, -0.014833208, 0.010226098, -0.008830933, -0.041640304, 0.03590633, -0.00034136503, 0.0032176946, -0.045933116, 0.010218432, 0.008685284, 0.018995706, 0.012242188, -0.026615452, -0.034802463, -0.0018416942, 0.026569458, 0.0023284687, 0.0372555, 0.0023840454, 0.026032856, -0.031644177, 0.026247496, 0.005116882, -0.008110354, -0.009789151, -0.011437285, 0.054089464, -0.0034323353, -0.046301074, -0.009099234, -0.017646534, -0.022951227, -0.009781485, 0.0057569714, 0.0068723364, 0.020835483, -0.043786712, -0.0018934379, -0.0016423849, 0.0105863875, 0.014066634, 0.025511585, -0.0063395677, 0.0017535381, -0.023748465, -0.0020275884, -0.021893356, 0.015699437, -0.0074970946, -0.009382866, 0.031046249, 0.030233681, 0.023119874, 0.006186253, -0.01474122, -0.029743074, 0.004350308, 0.0048332494, -0.0036891378, 0.0013165909, -0.0125718145, 0.022797912, -0.004177829, 0.034587823, 0.011835904, 0.002405126, -0.008823267, 0.03612097, -0.016849298, 0.0054886704, -0.0036527254, 0.021065455, -0.021694046, -0.023089211, -0.015653443, 0.021847362, -0.027397357, -0.011943224, 0.034311853, -0.019792942, 0.023135206, 0.028025948, -0.012970434, 0.052740294, -0.017661866, 0.0072326264, 0.012134868, -0.0074089384, -0.013384383, 0.00786505, 0.013928651, 0.0021329923, -0.003443834, -0.018919049, -0.016006067, -0.0019873432, 0.0075814174, 0.006094264, 0.02368714, 0.026339484, 0.014503581, -0.015392807, -0.01464923, 0.015676439, 0.015400473, -0.013445709, 0.012740461, 0.041946933, 0.029635753, 0.014572573, -0.0066921916, -0.020559516, -0.038144726, 0.025848877, 0.01997692, 0.0005763679, 0.0064928825, 0.006906832, -0.016328027, -0.006738186, 0.0019490146, 0.025710894, -0.0053353556, -0.023411172, 0.009390532, 0.008301998, -0.0066385316, 0.01790717, -0.00673052, 0.0038060402, -0.0046607703, 0.002211566, 0.0055653276, 0.009620504, -0.005584492, 0.005412013, -0.009513184, -0.02773465, 0.010670711, 0.015615114, -0.009566844, 0.016297365, -0.027474014, -0.03464915, 0.006205417, -0.010111112, -0.0045956117, -0.013920985, 0.039279256, -0.008118019, -0.010992672, 0.009237218, -0.023211863, -0.050501898, 0.04998063, -0.016266702, 0.029129814, -0.0016232206, 0.003990018, -0.035477046, 0.029099151, -0.017155927, -0.042805497, 0.039248593, -0.013683347], "content_hash": "856da370f89a42dffbfd36bb5e605de3"}, {"content": "# Tokenization Explained\n\n## What Is Tokenization?\n\nTokenization is the very first step in how a language model processes your text. Models cannot read raw text -- they need it broken into small, numbered pieces called **tokens**. Tokenization converts your input string into a sequence of these tokens.\n\n## Why Not Just Use Words?\n\nYou might wonder why we don't just split text by spaces. The problem is that there are too many possible words (including misspellings, rare terms, and words in other languages). Instead, modern models use **subword tokenization**, which breaks text into smaller, reusable pieces.\n\nFor example, the word \"unhappiness\" might become three tokens: \"un\", \"happiness\", or \"un\", \"happ\", \"iness\" -- depending on the specific tokenizer.\n\n## How It Works: BPE\n\nMost models (including GPT-2 and LLaMA) use a method called **Byte-Pair Encoding (BPE)**. Here's the intuition:\n\n1. Start with individual characters as your vocabulary\n2. Find the most common pair of adjacent characters in the training data (e.g., \"t\" + \"h\" = \"th\")\n3. Merge that pair into a new token\n4. Repeat thousands of times\n\nThis builds a vocabulary of common subwords. Frequent words like \"the\" become single tokens, while rare words get split into pieces.\n\n## Token IDs\n\nEach token has a unique **ID** -- a number that the model uses internally. For example, in GPT-2's vocabulary, the token \"the\" might have ID 262, while \"cat\" might have ID 9246. The model never sees the text itself; it only works with these IDs.\n\n## What You See in the Dashboard\n\nIn **Stage 1 (Tokenization)** of the pipeline, you can see exactly how your input text was split:\n\n- Each row shows a **token** (the text piece) and its **ID** (the number)\n- The summary shows the total number of tokens\n- Notice how spaces are often attached to the following word (e.g., \" cat\" with a leading space is one token)\n\nThis stage helps you understand that the model's \"unit of thought\" is the token, not the word.", "source_file": "tokenization_explained.md", "chunk_index": 0, "embedding": [0.0115930205, 0.003369542, 0.014504099, 0.0053508715, 0.011047995, -0.05714436, -0.023327107, 0.054784715, 0.008752474, 0.05365619, 0.009829702, -0.001070014, -0.026417723, -0.060581226, 0.008502403, -0.01112494, 0.002255445, 0.03216294, -0.015799338, 0.05427175, 0.02296803, 0.0045012725, -0.015504383, 0.051270902, 0.019903062, -0.008021498, -0.00911155, 0.011298065, -0.0131319165, -0.012574067, -0.013901365, -0.016761148, -0.034445632, -0.0341635, 0.018697593, -0.0073161707, -0.03998566, 0.020377554, 0.019210558, 0.027597545, -0.051193956, -8.9819056e-05, -0.051501736, 0.04803922, -0.051655628, 0.048859965, 0.026520317, -0.03285544, -0.03034191, 0.014093727, -0.032598957, 0.060427338, -0.022108814, -0.024789058, -0.013503817, -0.010175954, -0.023032151, 0.010220838, -0.046346433, -0.05991437, 0.020133896, 0.00951551, 0.045756523, 0.028828662, 0.009829702, -0.020967465, -0.044730593, 0.003821593, -0.02148043, 0.023301458, 0.07099443, 0.031111358, -0.019851765, 0.026545964, 0.0070468634, -0.03524073, -0.05070664, 0.048603483, -0.021865154, -0.035907585, 0.030906172, -0.064171985, -0.011721262, -0.049142096, 0.015812162, -0.010951813, -0.038780194, 0.0029671849, -0.012683072, -0.04773144, -0.023763128, 0.012484298, -0.052578963, 0.00805997, 0.026033, -0.023634886, -0.029315978, -0.025019892, -0.014632341, -0.008015086, -0.0003716996, -0.0033150397, -0.020634037, -0.020710982, 0.027187172, -0.061453268, 0.012958791, -0.0014795848, -0.045961708, -0.04001131, -0.10249051, 0.03275285, 0.0016527107, 0.031342193, 0.055348977, -0.013349927, 0.0061203195, 0.03418915, 0.014734934, -0.05888844, -0.003056954, -0.04580782, 0.010188777, -0.020749455, -0.026597261, -0.0012399338, 0.020108247, 0.0067583206, -0.006069023, -0.022455065, 0.0039049499, 0.01456822, 0.017158696, -0.009816878, -0.010560677, -0.050347563, -0.019043844, 0.033573594, -0.01382442, 0.0026273453, 0.0014555396, -0.043473825, 0.031726915, -0.017774254, -0.06653163, -0.046038654, -0.020993114, 0.060324743, -0.07078924, -0.009150022, 0.0017264495, -0.022390945, 0.0025920789, -0.019082317, 0.009015368, -0.042627435, -0.00058069295, 0.049013853, 0.021557376, -0.01494012, -0.008874304, 0.024994245, 0.011490427, 0.010105421, -0.046038654, 0.060632523, 0.0014274868, 0.011580196, -0.032983683, -0.059657887, -0.01719717, -0.03303498, -0.025532858, -0.0431404, -0.026956337, -0.0048058457, -0.05883714, -0.038908433, -0.0047192825, -0.0833056, -0.035958882, -0.011182648, -0.054220453, 0.010201601, -0.018453933, -0.008912776, -0.026597261, 0.040242143, -0.029700702, 0.025866285, -0.0036580854, 0.027623191, 0.03542027, 0.010432436, 0.0032172555, -0.013516641, 0.0453718, -0.020198017, 0.0035298439, 0.032496363, 0.02482753, 0.03629231, 0.0045237145, -0.00052418665, 0.0363949, 0.0059279576, -0.03608712, 0.003629231, -0.011811031, -0.032188587, -0.012772841, -0.04550004, 0.016299479, -0.030111074, -0.07058405, 0.02841829, -0.033855725, 0.047962274, 0.029777648, 0.017312586, 0.03911362, 0.010022064, 0.04088335, 0.004408297, -0.038164634, -0.008566524, 0.047218475, -0.03480471, 0.012695896, -0.011612257, -0.0032429039, 0.012612539, 0.0074123517, -0.03226553, -0.011208296, -0.034727763, 0.01323451, -0.034009613, -0.007540593, -0.029264683, -0.023070624, 0.01286261, 0.026186889, 0.03282979, 0.02988024, -0.007848372, -0.023840072, -0.03511249, 0.052245535, 0.034881655, 0.037420835, 9.6130934e-05, -0.06976331, 0.027828379, 0.0014627532, 0.034522578, 0.0193388, 0.027751435, 0.032958034, -0.0088678915, 0.02290391, -0.035907585, -0.06068382, -0.018107682, -0.029777648, -0.04488448, -0.042730026, -0.0023548324, -0.010977462, -0.042909563, -0.008848655, 0.0061235256, 0.013619234, -0.014619517, -0.038138986, -0.012356057, -0.042396598, -0.024276093, -0.047987923, 0.030367557, 0.011278829, -0.015568503, 0.075611115, 0.007091748, -0.011875152, -2.4521154e-05, 0.023391226, 0.016568786, 0.0060497867, -0.010060536, 0.027623191, 0.01233682, 0.029341627, 0.0123432325, 0.0026545965, 0.021647144, 0.053912673, 0.01850523, -0.0015557281, -0.013401223, 0.03939575, -0.08227967, -0.017594717, -0.028597826, 0.026828095, -0.0066813757, -0.0061908523, 0.016902214, -0.028366992, -0.030624041, -0.014363034, -0.050527103, -0.005713153, -0.026494669, 0.015914755, -0.0053508715, 0.042781323, -0.0014587457, 0.016402071, -0.0032124466, -0.003808769, -0.030829227, -0.0064761895, -0.019633755, 0.019056669, 0.0040588398, -0.032701552, 0.0075534172, 0.012195755, -0.038164634, 0.04968071, 0.006886562, -0.025481561, 0.025430264, -0.042037524, -0.025866285, 0.025558507, 0.035753697, -0.0038408292, 0.06350513, -0.02841829, -0.04614125, -0.018864306, -0.008188212, -0.05324582, 0.026571613, 0.04547439, -0.0104067875, 0.028649123, 0.014145024, -0.0074957083, -0.00068208383, 0.010688919, -0.055708054, -0.014298913, 0.008277981, 0.011945684, -0.02839264, 0.028777365, -0.013657707, -0.020492971, 0.0003664898, 0.026905041, 0.039626583, -0.009400093, -0.012477886, -0.017338235, 0.009880998, -0.039523993, -0.016479017, -0.0123817045, -0.010195189, 0.025366144, -0.0053412532, -0.020044127, 0.0040716636, 0.008893539, 0.027110226, -0.021095706, 0.031573027, 0.045756523, 0.0033342757, -0.029367276, 0.00059271563, 0.006803205, 0.036625735, 0.02014672, 0.0073033464, -0.038010743, 0.0016623288, -0.028213102, -0.006264591, -0.0026898629, 0.047910977, 0.020377554, 0.0497833, -0.05334841, 0.026725503, 0.022480713, 0.0049757655, 0.016568786, 0.0050719464, -0.031137006, -0.043012157, 0.02662291, 0.011599433, 0.025071189, 0.028520882, 0.033060625, 0.04814181, 0.029854592, 0.028469585, 0.017030455, -0.026981985, -0.01174691, 0.01701763, 0.014055255, 0.03957529, 0.015196603, 0.023840072, -0.010554265, -0.04088335, -0.054323047, -0.039216213, -0.04355077, 0.004302498, 0.01651749, -0.004786609, -0.007547005, -0.016786797, 0.019043844, 0.011484015, 0.03611277, -0.04057557, 0.0008287599, -0.027854027, 0.01605582, 0.050168026, -0.0007662422, 0.012407353, -0.0008311644, -0.018300043, -0.017876849, -0.0048026396, -0.061761048, 0.0022891085, -0.013272982, -0.07032757, 0.029033847, -0.027623191, 0.013991134, 0.02113418, 0.014606693, 0.038626302, -0.008303629, -0.08453672, 0.037805557, -0.008008674, 0.027469303, -0.011734086, 0.03865195, 0.00064120686, 0.00033282646, 0.020108247, -0.0073289946, 0.0070661, -0.036651384, -0.018248748, 0.03536897, -0.009842525, -0.012163695, -0.0072328136, 0.011163412, -0.042627435, -0.02929033, -0.02764884, 0.032624606, -0.0057227714, -0.018415462, -0.0026610086, 0.042294007, -0.0051232427, -0.025596978, 0.020390378, 0.007361055, 0.08756321, 0.0011934463, -0.011541724, 0.0014194717, -0.03216294, -0.027854027, -0.03318887, 0.019595282, -0.013157565, -0.011086467, 0.011471191, -0.03213729, -0.04062687, 0.01435021, 0.00039313996, -0.016094293, -0.011753323, -0.00021039601, 0.06955812, 0.01955681, -0.0005766854, 0.035599805, 0.01208675, 0.0008928806, 0.0150298895, -0.052835446, -0.03895973, 0.021070058, 0.017415179, -0.024647992, 0.022891086, 0.0041518146, 0.000885667, 0.0067454963, -0.00167355, 0.01962093, 0.035907585, 0.039652232, -0.035856288, -0.0121829305, -0.013939838, -0.042755675, -0.026443372, -0.03403526, 0.01162508, 0.026058648, -0.021211123, 0.011407071, -0.00013164779, -0.035266377, 0.012041865, -0.0009217349, -0.019415744, -0.03734389, 0.011028758, 0.012599715, 0.008720414, 0.014414331, 0.002983215, 0.0018146154, 0.00805997, 0.016376425, -0.022019044, -0.008284393, 0.0379338, 0.026853744, -0.004558981, 0.03447128, -0.037600372, -0.033471, -0.010945401, 0.015260723, -0.01605582, -0.0057355957, -0.021365013, -0.010387551, 0.0008223478, 0.038626302, -0.021608671, -0.020813575, 0.060581226, 0.023327107, -0.012465062, 0.022480713, -0.015491558, -0.026699854, -0.0046936343, 0.0047289007, 0.0069378586, -0.008835831, -0.028649123, 0.015209427, 0.013106269, 0.0035811404, 0.021044409, -0.023942664, 0.004911645, -0.0047930214, 0.005597736, -0.04801357, 0.022211406, 0.01494012, -0.025584154, -0.030675337, -0.0003682932, 0.03611277, -0.038728897, -0.010278546, 0.045294855, -0.031752564, 0.016479017, 0.037087407, -0.021595847, -0.04937293, 0.0073546427, 0.007264874, -0.033676185, -0.040549923, 0.020480148, -0.04344818, 0.0073033464, -0.018646296, 0.038446765, 0.0063832146, 0.012157283, 0.0029255063, -0.035856288, -0.015286372, -0.016786797, -0.034856007, 0.03318887, -0.047038935, -0.023827247, 0.013260159, -0.019723523, 0.02485318, -0.018133331, -0.011054407, -0.05457953, 0.015799338, 0.045243558, -0.012715132, -0.026981985, 0.0026994809, 0.036625735, -0.015735216, 0.011214708, 0.05878585, -0.0013561526, -0.012849786, 0.018838659, 0.013452521, 0.025840636, -0.00561056, -0.029623758, 0.008342101, -0.0068737376, 0.0031082504, -0.0016687409, -0.021467606, -0.009425741, 0.008887127, 0.050039783, -0.0024734556, -0.028084861, 0.0061107017, -0.03672833, 0.05263026, -0.006713436, -0.0046006595, -0.016158413, -0.021082882, -0.01642772, 0.0017264495, 0.025160957, 0.0034208386, 0.016017348, -0.033240166, -0.011176236, -0.009925882, 0.046372082, 0.029392924, 1.7871136e-05, 0.0037125878, -0.014465627, -0.07971484, 0.032085992, 0.0019091935, -0.005187364, 0.012567655, 0.024622343, -0.006363978, -0.031983398, -0.0022073546, 0.024417158, -0.03642055, 0.0016863741, -0.033958316, -0.013952661, -0.012792077, -0.01992871, 0.050937474, -0.020621214, 0.02076228, -0.013247334, -0.0019797261, -0.037113056, 0.0026000938, -0.017658837, -0.025302023, 0.003731824, 0.001210278, 0.04131937, -0.0058542187, -0.038780194, -0.031008765, -0.021980572, -0.043781605, -0.025802165, -0.0070596877, 0.036779627, 0.002138425, 0.04283262, -0.010022064, -0.006052993, -0.043063454, 0.028751716, 0.024365861, 0.018556528, -0.006149174, -0.004212729, -0.025994526, 0.017274113, 0.008303629, -0.0020181986, -0.044525407, 0.016145589, 0.0073097586, 0.010688919, 0.02228835, -0.03939575, -0.026366428, -0.02972635, 0.00078107015, 0.017889671, 0.015453086, 0.045115314, -0.007463648, -0.038728897, 0.00035627058, 0.015414613, -0.0055496455, -0.00017583095, -0.034317393, -0.00028874347, -0.021595847, 0.02163432, -0.014965769, 0.012766429, 0.008714002, 0.0016182458, -0.017133048, 0.004934087, -0.011657141, -0.04180669, -0.0017200374, 0.010535029, 0.0019123995, -0.015863458, 0.017774254, 0.0013850068, -7.864803e-05, 0.007117396, -0.017722959, 0.028033564, -0.0066942, -0.0023836866, -0.0021047615, 0.027110226, -0.0074059395, -0.007720131, 0.005491937, 0.009226967, 0.033522297, -0.0011685996, -0.028443936, -0.0098938225, 0.014901648, -0.008976896, 0.052886743, -0.021544551, 0.04134502, 0.0076880706, 0.0038151809, -0.014260441, -0.028597826, 0.019941535, 0.0059087216, -0.03093182, 0.0069506825, -0.0027892499, -0.043037806, -0.0032092405, -0.021749737, -0.030239316, -0.02340405, 0.007393115, 0.022185758, -0.017530596, -0.028290048, 0.01419632, 0.002888637, -0.08966637, -0.021711266, 0.026148416, 0.010490145, 0.008104855, -0.024032434, 0.036574442, 0.012580479, -0.02485318, -0.04001131, 0.015055537, -0.013061384, 0.014478452, -0.01999283, 0.0006103488, -0.0267768, 0.022570482, 0.018838659, 0.0011902403, 0.0062068827, -0.046372082, 0.0089448355, 0.024660816, 0.0067519085, -0.023840072, -0.005495143, -0.008637057, 0.032342475, -0.0016591228, 0.005681093, -0.022211406, 0.022275526, -0.035292026, -0.011484015, 0.01456822, 0.049937192, -0.017133048, 0.0007550211, 0.022083165, -0.026853744, 0.036959164, -0.015696744, -0.015158131, -0.011298065, 0.029777648, -0.040652517, -0.009528334, 0.005693917, -0.031111358, 0.034932952, -0.032316826, 0.026212538, 0.01118906, -0.0019011784, 0.03762602, 0.002138425, 0.011977744, -0.036830924, 0.022275526, 0.048116162, 0.0012014614, 0.019018196, -0.019249031, 0.023173217, -0.0052001877, 0.0063960385, 0.014260441, 0.03795945, -0.031290896, -0.022275526, -0.016773973, 0.008348513, 0.023737479, -0.0073289946, 0.01267666, 0.023327107, -0.012138046, 0.028828662, -0.022108814, -0.026417723, 0.024596697, -0.005786892, -0.016838092, 0.03675398, 0.028597826, 0.03303498, 0.0035009896, -0.024635168, -0.013452521, 0.009291087, 0.024455631, 0.002503913, -0.030726634, -0.022621779, -0.018171802, -0.0043698247, 0.0075277686, 0.046346433, 0.022826964, -0.0151068345, 0.048731722, 0.023314282, -0.047526255, -0.02333993, -0.015068362, -0.017812727, 0.040088255, -0.035035543, -0.017876849, -0.020069776, 0.016132765, 0.023186041, -0.012240639, 0.027725786, -0.07012238, 0.04355077, 0.06232531, 0.032598957, -0.0026754357, -0.027700137, -0.023147568, -6.3619746e-06, 0.0060946713, 0.031034414, -0.041396316, 0.008072794, -0.020480148, 0.062479198, -0.0078098997, -0.0038055629, 0.0016390851, 0.03121395, -0.003642055, -0.041550204, -0.026699854, 0.024429983, -0.017312586, 0.02885431, -0.016761148, 0.001126921, -0.025225079, -0.0016054218, 0.0010579914, 0.015286372, -0.012548419, 0.009759169, -0.013247334, -0.034907304, 0.000559453, 0.02253201, 0.008758886, 0.033983964, -0.0130549725, 0.027264116, -0.024429983, -0.009996415, 0.00243338, -0.048552185, -0.0252892, 0.04180669, 0.0028677979, 0.02098029, 0.016325127, 0.0022971237, 0.03506119, -0.01776143, -0.012958791, 0.031034414, -0.03288109, 0.007822724, -0.008996133, 0.022237055, 0.016966334, 0.015004241, 0.016722675, 0.0070468634, -0.00926544, 0.014850351, 0.01620971, 0.015889106, -0.031419136, 0.020852048, -0.013311455, -0.024224795, -0.0070340396, -0.028879957, -0.022198582, -0.007559829, -0.030906172, 0.01583781, 0.03377878, 0.010336255, 0.03003413, 0.0129331425, -0.010034888, -0.0126446, 0.020903345, 0.003542668, -0.015132482, 0.02333993, -0.02513531, 0.028033564, 0.018441109, -0.014657989, -0.018992547, 0.010522204, 0.0013248937, 0.010438848, 0.0197107, 0.0034849595, 0.0046295137, 0.021300893, -0.019172085, 0.043730307, -0.016786797, -0.024776233, 0.007944553, -0.014055255, 0.026545964, -0.017261289, -0.056733985, -0.018338516, -0.0033887783, 0.0034945775, 0.00067847705, -0.0076880706, 0.02616124, 0.012529182, -0.021147003, -0.0075726532, -0.032778494, -0.014311737, -0.0103683155, -0.00035106076, 0.015914755, 0.042140115, 0.013503817, -0.012727956, 0.021685617, 0.010637622, -0.002980009, 0.0234297, -0.016107116, -0.0121444585, -0.007239226, -0.011413482, 0.02340405, 0.020531444, 0.0033855722, -0.03344535, 0.044961426, 0.030598393, 0.012509946, -0.030470151, 0.034856007, -0.0048924084, 0.049860246, 0.03747213, 0.01553003, -0.021608671, 0.02913644, 0.029059496, 0.052886743, 0.0038376232, -0.0130549725, -0.0062838276, 0.0017857612, 0.010483732, 0.042704377, -0.012471474, -0.033265814, 0.043781605, 0.00075261656, 0.0003885313, 0.009900235, -0.008893539, 0.017889671, -0.014491276, 0.009624516, -0.0043441765, -0.012048277, -0.009643751, 0.009964355, -0.0016879771, 0.012247051, -0.0029271094, -0.002519943, -0.01912079, 0.0113942465, -0.011266005, -0.004841112, -0.013208861, 0.01865912, 0.026545964, -0.008880715, 0.008412634, 0.0018338517, 0.00016611266, -0.0135294655, -0.039190564, 0.007393115, -0.0178512, 0.010560677, -0.010829984, -0.0038023568, 0.023173217, 0.033240166, -0.004302498, -0.010201601, 0.0009209334, 0.00836775, -0.0013826024, 0.012317584, 0.034958597, -0.0051969816, -0.023699006, 0.0111569995, -0.040036958, 0.04342253, -0.031701267, -0.020018479, 0.0092205545, -0.017684486, 0.02631513, 0.022365296, 0.0014507306, 0.036830924, -0.011304477, -0.0053701075, 0.025417442, 0.053297117, -0.007918905, 0.039652232, 0.017248465, -0.047680143, -0.03554851, -0.030290613, -0.019313151, 0.002151249, -0.03223988, 0.019236207, 0.022814142, -0.012061101, -0.008726826, 0.021890802, 0.0012383307, 0.003728618, -0.010509381, 0.040960297, -0.013452521, 0.052502017, -0.03211164, -0.025096837, -0.0013088635, 0.00083276746, -0.00373503, 0.00691221, -0.012157283, -0.034266096, 0.0067519085, 0.01592758, -0.0061235256, 0.002229797, -0.015299196, -0.0060497867, 0.019390097, 0.009611691, 0.010573502, 0.06514662, 0.0006215699, 0.013593586, 0.009861762, -0.017351057, -0.018094858, -0.017864024, -0.005943988, 0.0009858556, -0.0017024042, 0.021608671, -0.0014884014, -0.023327107, -0.040909, -0.01044526, 0.020852048, -0.018133331, 0.012869022, -0.005293163, 0.017209992, -0.017979441, 0.0053284294, 0.01007336, 0.0039690705, -0.0024478072, 0.0100925965, -0.004087694, 0.031983398, 0.021801034, -0.017594717, -0.020659685, -0.0077073066, 0.024994245, -0.0027251292, -0.015889106, -0.010060536, 0.02839264, -0.03106006, 0.0078034876, -0.0074828845, 0.009752757, -0.00633833, -0.0208264, 0.032958034, -0.015055537, 0.0002763201, 0.018825835, -0.040344737, -0.016607258, -0.0068480894, -0.018800186, 0.028751716, -0.025686748, -0.016158413, -0.053707488, -0.014247617, 0.015504383, -0.019595282, -0.01032343, 0.00042159352, 0.004469212, 0.007168693, -0.025237903, -0.00982329, -0.017966617, 0.027546247, 0.013619234, -0.011676378, 0.024417158, -0.033650536, -0.0034849595, 0.0245454, 0.012439414, 0.013516641, -0.00020879299, 0.04480754, -0.006341536, -0.026597261, -0.03731824, -0.016697027, 0.0008095237, -0.022147285, 0.012202167, -0.0074123517, -0.011413482, 0.024263268, 0.013593586, -0.036343604, -0.037574723, 0.010836396, 0.014439979, 0.009169258, 0.0013329088, 0.012638188, 0.003818387, -0.014581044, 0.0031932103, 0.043319937, -0.008175388, 0.064171985, -0.015376141, -0.004533333, 0.02141631, -0.027854027, 0.018928427, 0.015042714, 0.03911362, -0.0008163365, 0.020903345, 0.0072328136, -0.029700702, -0.024647992, -0.00052098057, 0.017133048, 0.0035266378, -0.041755393, -0.022634603, -0.010746627, -0.016991982, 0.00012423383, -0.028033564, 0.05935011, 0.007091748, -0.008951248, -0.03403526, -0.00023965107, 0.039831772, 0.007906081, 0.01837699, -0.003917774, -0.033291463, -0.0026481843, -0.01047732, -0.021749737, -0.0042479956, 0.016466193, 0.02392984, -0.028495234, 0.0032124466, 0.00991947, -0.004757755, 0.017286938, 0.023699006, 0.0042383773, -0.0204545, -0.007765015, 0.0021272039, 0.014170672, -0.005943988, 0.010746627, 0.027341062, 0.04521791, -0.00709816, -0.008701177, -0.010708155, 0.05098877, 0.021852331, 0.0029976422, 0.008681941, 0.015286372, 0.0042319656, 0.0011124939, -0.028777365, 0.0055272034, -0.0021897214, -0.030239316, -0.01791532, 0.01955681, -0.023634886, -0.014042431, -7.62936e-05, -0.016684202, 0.016863741, -0.048706073, -0.029675055, -0.003443281, 0.0103683155, 0.011939272, 0.012131634, 0.012009805, 0.04593606, -0.031624325, -0.0034657232, -0.0121829305, -0.016979158, -0.023006503, -0.031239599, 0.036523145, -0.02990589, -0.016235359, -0.015453086, -0.006389627, -0.02489165, -0.004036397, 0.00093776506, -0.0018691181, 0.02358359, -0.009047429, -0.031239599, -0.017787078, 0.010676094, -0.04419198, 0.052732855, -0.010150305, 0.047346715, -0.023391226, -0.018312868, -0.009810465, -0.022467889, 0.01066327, 0.029341627, -0.0163636, 0.017030455, -0.022698723, 0.0074957083, 0.03508684, -0.0051841578, 0.0038696835, -0.008092031, 0.011272417, 0.0050943885, -0.013747475, -0.023237336, 0.018620647, 0.0051488914, 0.0033471, -0.005963224, 0.039190564, -0.021300893, 0.008310041, -0.014439979, -0.022519186, 0.02489165, 0.028315695, -0.015966052, -0.04652597, -0.011881564, 0.0036099947, 0.00633833, 0.02349382, 0.0021768974, -0.016863741, -0.02340405, 0.016299479, 0.0126189515, 0.004687222, 0.016299479, 0.02051862, 0.025019892, -0.03303498, -0.03967788, 0.038626302, -0.006963507, -0.01819745, -0.0079958495, -0.010355491, 0.017658837, 0.05873455, 0.026699854, -0.0021929275, 0.017864024, -0.015081186, -0.0052066, -0.009002545, 0.025956055, -0.0049885893, 0.025109662, 0.0031403108, 0.0098938225, 0.013260159, -0.0005911126, -0.014632341, 0.0063062697, 0.0047160764, -0.003728618, -0.006969919, -0.004456388, 0.022237055, 0.019069493, -0.0026578025, -0.0072007533, 0.04742366, -0.012163695, -0.019633755, 0.025622627, -0.013709003, 0.025109662, 0.011849503, 0.0018130124, 0.016620083, 0.008201036, -0.012734368, -0.0051649213, -0.02209599, -0.040242143, -0.011144175, 0.0006219707, -0.0048282878, -0.025981702, -0.020274961, 0.02014672, 0.03631796, 0.007341819, 0.03434304, 0.0014659592, -0.013414048, 0.025391793, -0.041704096, 0.0002630952, 0.01242659, 0.024558224, 0.015953228, 0.007617538, 0.034599524, 0.04265308, 0.002431777, 0.02200622, 0.0155813275, -0.013022912, -0.0014835924, 0.036471847, -0.010862045, -0.026174065, -0.0046776044, 0.011669965, -0.019364448, 0.010515793, 0.07243073, -0.01509401, 0.0042544077, 0.0070404517, 0.02172409, 0.012349645, 0.029700702, 0.012311172, -0.028084861, -0.009957943, 0.02929033, 0.004366619, 0.00307619, -0.006886562, -0.055554163, -0.054938607, -0.010932577, 0.0055464394, -0.0007498113, 0.006056199, 0.0030104662, 0.00078708143, -0.05529768, -0.003818387, 0.028931255, -0.0029303154, -0.00429288, 0.0018370578, 0.017889671, 0.012234227, 0.009425741, -0.02749495, -0.00023544315, -0.011080055, 0.0056233844, 0.042165764, 0.0013321073, 0.021095706, 0.0012896273, 0.037677318, 0.007085336, 0.013349927, 0.0022233848, 0.027777081], "content_hash": "5a2eb44b34de90d11581ea0506ca1750"}, {"content": "# The Transformer Architecture\n\n## What Is a Transformer?\n\nA **Transformer** is the specific type of neural network architecture used by modern LLMs like GPT-2, LLaMA, and others. It was introduced in the 2017 paper \"Attention Is All You Need\" and quickly became the dominant approach for language tasks.\n\n## The Key Innovation: Attention\n\nBefore Transformers, language models processed words one at a time, left to right. Transformers changed this by introducing the **attention mechanism**, which lets the model look at all words in the input simultaneously and figure out which ones are relevant to each other.\n\nFor example, in \"The cat sat on the mat because it was tired,\" attention helps the model connect \"it\" back to \"the cat.\"\n\n## How Layers Stack\n\nA Transformer is built from identical **layers** stacked on top of each other. Each layer has two main parts:\n\n1. **Attention**: Looks at relationships between all tokens\n2. **MLP (Feed-Forward Network)**: Processes each token's information individually, retrieving stored knowledge\n\nA small model like GPT-2 has 12 layers. Larger models may have 32, 64, or more.\n\nInformation flows through these layers sequentially. After each layer, the model's understanding of the text becomes more refined. Early layers tend to capture basic patterns (like grammar), while later layers capture more complex meanings.\n\n## Encoder vs. Decoder\n\nThe original Transformer had two halves:\n\n- **Encoder**: Reads and understands the full input (used in models like BERT)\n- **Decoder**: Generates text one token at a time (used in GPT-style models)\n\nMost LLMs you'll encounter in this dashboard are **decoder-only** models. This means they generate text left-to-right, predicting one token at a time based on everything that came before it. Each token can only \"see\" the tokens to its left -- it cannot look ahead.\n\n## The Residual Stream", "source_file": "transformer_architecture.md", "chunk_index": 0, "embedding": [-0.0018053533, 0.028315073, 0.008433898, -0.021741506, 0.060291372, -0.010371495, -0.0030208095, 0.034187302, -0.04341169, 0.06357221, 0.00401784, -0.024071379, -0.032903492, -0.015881166, -0.016023811, -0.025961427, 0.035019398, -0.00904014, 0.018020844, 0.06276389, -0.005800905, -0.036208108, 0.018520102, 0.045812886, 0.018591424, 0.009265995, -0.024225911, 0.07874015, -0.024463654, -0.015643423, 0.0021367064, -0.016606279, -0.070133895, 0.018460667, 0.013254117, 0.0099495035, 0.022989653, 0.046383467, -0.0017815792, 0.0535395, 0.013598843, -0.004531957, -0.039798014, 0.032285366, -0.034282397, 0.012231827, -0.010591406, 0.0037949567, -0.03694511, 0.060719308, -0.011043116, 0.03756324, -0.039013464, -0.00018332138, -0.012540891, -0.011827664, -0.04141466, 0.01340865, -0.017533474, -0.024154589, -0.00020338086, 0.0056107114, 0.019780135, 0.027364105, 0.018829167, -0.005949494, -0.05886492, 0.040273495, -0.023013426, 0.02926604, 0.050068468, 0.02102828, -0.050971888, -0.008879664, 0.017260069, 0.008083228, -0.04291243, 0.07992886, -0.0050817356, -0.019007474, 0.011833608, -0.039346304, 0.0052422113, -0.01926899, -0.012303149, 0.021159038, -0.0050133844, -0.030573621, -0.039536495, -0.0624786, 0.003170884, -0.0074591553, -0.05615466, 0.015072843, 0.024487427, 0.029836621, -0.018175377, -0.0045379004, -0.018852942, 0.039132334, 0.04329282, -0.0010921274, 0.005167917, -0.030597396, -0.011910874, -0.035661303, 0.015726633, -0.03989311, -0.068517245, -0.027625622, -0.07588725, -0.030121913, 0.009937616, 0.017402714, 0.047310658, -0.038680624, 0.010591406, 0.031952526, -0.01918578, -0.00064413226, -0.011572092, -0.020719215, 0.00969393, 0.033260107, -0.0036523116, 0.0035364123, 0.0013462141, -0.0005527502, 0.019982215, 0.028101105, -0.001479944, 0.029955493, 0.024178362, -0.067423634, -0.008315027, -0.017676119, -0.05063905, 0.014454714, -0.030930236, 0.016214006, 0.026674652, -0.03922743, -0.009563172, 0.01955428, -0.0379674, 0.015477004, 0.011643414, 0.030644944, -0.05705808, 0.022312088, 0.019815797, 0.02054091, -0.058722276, -0.020731103, -0.024273459, -0.021824716, 0.009242221, 0.07807448, 0.046217047, -0.027696945, 0.0030995614, 0.034377493, -0.0078870915, -0.0040743034, -0.034282397, 0.051922854, -0.005429433, -0.0015304642, -0.042864885, -0.059387952, -0.024011943, -0.016416086, -0.028291298, 0.010591406, 0.014621134, -0.022145668, -0.07912054, -0.04940279, -0.0030460695, -0.047310658, -0.022193216, -0.0010208047, -0.045646466, -0.012142673, -0.047548402, 0.008731076, -0.017188746, 0.05391989, -0.011560205, 0.0069598975, -0.022704361, 0.05800905, 0.05025866, 0.028433945, 0.009105518, 0.008469559, 0.054823305, -0.024677621, 0.059863437, -0.025557267, 0.04198524, 0.025795007, 0.01988712, -0.049545433, 0.04086785, 0.020029765, -0.02033883, 0.016225891, -0.030716268, -0.048023887, 0.017640457, -0.042936206, -0.0010802402, -0.01397923, -0.014003004, -0.032309137, -0.045147207, 0.01947107, 0.0028187288, 0.028029783, 0.03135817, 0.033426527, -0.007958414, 0.018829167, -0.061527632, 0.017545361, 0.01873407, -0.042936206, 0.006424978, 0.020719215, 0.017806876, -0.0093729785, -0.00079643575, -0.040653884, -0.009747422, -0.048927307, 0.029646428, 0.0069598975, 0.024332894, 0.001638934, -0.02574746, 0.013075811, -0.0039970377, 0.007417551, 0.030121913, 0.015845506, -0.010829149, -0.046644982, 0.05800905, 0.058769826, 0.032023847, 0.0010260054, -0.038514204, 0.011756342, -0.030740041, 0.025842557, -0.00084621296, 0.007078768, -0.023595894, 0.017521586, 0.035922818, 0.04471927, -0.014668682, -0.0048677674, -0.01287373, -0.0312393, -0.0189837, -0.026627105, -0.0076196315, -0.050211113, 0.022205103, -0.009616665, -0.014014891, 0.029361138, -0.05182776, -0.04046369, 0.02909962, 0.010739995, -0.002471031, 0.056725245, -0.004650828, -0.02795846, 0.04714424, 0.05539389, -0.009682043, 0.020422038, -0.031334396, 0.008826172, 0.0044487473, -0.019494845, 0.04215166, -0.0096761, 0.035399783, -0.021907926, -0.014870763, 0.024321007, 0.022763798, 0.036564723, 0.014740004, -0.026436912, -0.0046211104, -0.048000112, -0.014038665, 0.02099262, 0.003209517, 0.026698427, -0.04189014, -0.029955493, -0.0067221555, -0.0016686517, 8.488133e-05, -0.034924302, 0.027554298, 0.012564665, -0.0024338837, 0.03768211, 0.005049046, -0.027530525, -0.005417546, -0.019364087, -0.032784622, -0.025224427, 0.007173865, -0.034543913, 0.017890086, 0.03254688, -0.021206588, -0.010365551, -0.0018826196, -0.015060957, 0.05734337, -0.01361073, -0.016332876, 0.021586975, -0.047334433, -0.009188728, -0.020315055, 0.070656925, -0.026627105, 0.0005936121, -0.011245197, -0.003028239, -0.037824754, 0.002445771, -0.05449047, 0.046383467, 0.016784586, -0.0019658292, 0.028909428, 0.0042734127, -0.044006046, -0.031167977, -0.00562557, -0.040321045, -0.0062050666, 0.03780098, 0.009325431, 0.009105518, 0.028743008, -0.016939119, -0.03361672, -0.003000007, 0.024582524, 0.0046270536, -0.05301647, 0.0063595986, 0.010086205, 0.036255658, 0.013004487, -0.012909391, 0.047762368, 0.031334396, 0.0038098157, 0.0125884395, -0.004546816, -0.004546816, -0.00338188, -0.001124074, -0.023239281, 0.03152459, 0.015916828, 0.03238046, -0.0053194775, -0.008071342, 0.007494817, 0.046621207, -0.01096585, -0.014169424, -0.022942103, 0.021491878, -0.011156044, -0.03542356, -0.02693617, 0.035280913, 0.02172962, 0.01426452, 0.028101105, 0.009586946, 0.04652611, 0.03775343, 0.0032659809, 0.015227376, -0.027554298, -0.036469623, 0.030359654, 0.01590494, 0.04246072, 0.01890049, 0.030145686, 0.007690954, 0.07284415, -0.00024740028, -0.013266004, -0.018056506, -0.016000038, 0.002653795, 0.0268173, 0.0043387916, 0.0208262, 0.03131062, -0.025604814, -0.015488892, -0.03214272, -0.00731651, -0.020326942, 0.026698427, 0.010490366, -0.02619917, -0.023167958, -0.014193198, 0.013848472, -0.006870744, 0.02033883, -0.030858913, 0.012374472, -0.013479972, 0.017794989, 0.035138268, -0.00912335, -0.025604814, 0.04362566, -0.005286788, -0.008433898, -0.027411653, -0.03287972, 0.021063942, -0.009450245, -0.038347784, 0.0020579542, -0.036255658, 0.018627087, 0.012142673, 0.044077367, -0.014276408, -0.043102626, -0.024249684, 0.042603366, 0.043839626, 0.06504621, -0.025485944, 0.039584044, -0.031120429, -0.0068291393, -0.056677695, -0.023381926, -0.012124842, 0.024273459, -0.035613753, 0.011732568, -0.00684697, -0.008112946, -0.004977723, -0.0045349286, -0.01853199, 0.008956931, 0.010365551, 0.029218493, -0.00840418, 0.035043173, -0.064665824, 0.0020980733, -0.016653828, -0.029384913, 0.0004398227, -0.021586975, 0.0787877, -0.037491914, 0.007898978, -0.0010200618, -0.03627943, -0.010347721, -0.013670165, 0.025367072, -0.010971794, 0.033783138, -0.02529575, -0.049022403, -0.069753505, -0.016344763, -0.010222906, 0.016309101, -0.021254135, -0.005313534, 0.068469696, 0.024867814, 0.012517117, 0.034187302, 0.03865685, -0.0100327125, 0.0052659856, 0.006288276, -0.0086716395, 0.0028068416, 0.03775343, 0.00090564846, 0.036255658, 0.0060208165, 0.023619669, -0.0025215512, 0.0099495035, 0.013361101, -0.0009078773, -0.01397923, -0.0057979333, 0.021301683, 0.01926899, -0.035399783, -0.014205085, -0.054538015, 0.011257084, -0.0007570597, -0.021777168, -0.009093632, 0.035970367, -0.0536346, 0.005551276, -0.009569116, -0.04058256, -0.0759348, -0.02574746, -0.007572083, 0.02357212, 0.017069876, 0.010086205, 0.004609223, 0.013563181, 0.016023811, -0.045598917, 0.005218437, 0.033711817, 0.020671668, -0.036921334, 0.034900527, -0.033830687, -0.023096636, -0.007756333, 0.017355166, -0.0010936132, 0.0078454865, -0.020065425, 0.012635988, -0.0031768277, 0.019055022, -0.012719197, -0.024725169, 0.016404198, -0.013468085, 0.016855909, 0.032071397, -0.03967914, 0.0017904945, -0.013551295, 0.018757844, 0.010092149, -0.014121875, -0.0016359622, 0.01771178, 0.024321007, 0.0009762281, -0.002881136, -0.047334433, -0.029384913, -0.02636559, 0.0058603403, -0.036208108, 0.008451728, -0.032166492, -0.01434773, -0.04272224, 0.011102552, 0.04227053, -0.02357212, -0.005049046, -0.002230317, -0.023655329, -0.013230342, 0.029194718, -0.021384893, -0.057533566, 0.010466591, -0.0049064006, -0.0020237789, -0.028552815, -0.0069836713, 0.011138213, -0.0015260065, -0.020148635, 0.006924236, -0.015346247, -0.00475484, -0.022609266, -0.032118946, 0.00025092927, -0.014668682, -0.045717787, 0.026959945, -0.035613753, 0.0035601866, 0.03751569, 0.012350697, 0.017390829, -0.007043107, -0.017093651, -0.009188728, 0.013931681, 0.010187245, -0.021575088, -0.015286811, -0.034852978, 0.03005059, -0.032665752, 0.010002995, 0.066995695, 0.048618242, -0.010977738, 0.027863363, -0.015893053, -0.034662783, 0.032523107, -0.033664268, -0.03718285, -0.005497784, -0.020980733, -0.0018038675, -0.015797956, 0.020980733, -0.0011032715, 0.017022328, -0.016297214, -0.009301656, -0.017913861, -0.02172962, 0.058674727, 0.006448752, 0.023690991, 0.013432424, -0.011007455, -0.017509699, 0.005396744, 0.013551295, -0.050401304, -0.014121875, -0.018520102, -0.015370021, -0.031049106, 0.012208052, 0.0067578168, -0.0045289854, 0.02147999, -0.007679067, -0.08611015, 0.012707311, 0.01565531, 0.004784558, 0.008754849, -0.0146092465, 0.0026448797, -0.052255694, 0.007756333, 0.023928734, 0.004213977, 0.0226687, -0.031167977, -0.0105260275, -0.004460634, 0.006704325, -0.0133729875, -0.01910257, 0.034520138, 0.008808342, -0.019506732, -0.0064963005, 0.0045795054, -0.04374453, -0.0067756474, -0.013919795, 0.0053283926, 0.08572977, 0.012861842, -0.07973867, -0.030312106, -0.013384875, -0.05320666, 0.046454787, -0.010852923, -0.008142664, -0.012303149, 0.039370075, -0.008023793, -0.05715318, -0.04410114, 0.013741488, 0.029765299, 0.0056879777, 0.0005813535, 0.006199123, 0.00023829921, 0.00089376135, -0.0011575064, -0.01434773, -0.04098672, 0.0009687987, -0.01022885, 0.005670147, 0.012731085, -0.018757844, -0.04074898, 0.0005616655, -0.047239337, 0.016237779, 0.0030044646, 0.03078759, -0.002534924, -0.035328463, 0.0021991136, 0.00017877085, -0.024285346, -0.013860359, -0.039536495, -0.019043135, -0.037301723, 0.0040059527, -0.0040743034, 0.035685074, 0.011857383, 0.026222944, 0.0034294284, 0.021527538, -0.00053603394, -0.05063905, -0.022526056, 0.0075899134, 0.013551295, -0.011845496, -0.006187236, 0.013598843, 0.025890104, -0.023179846, 0.019364087, -0.016618166, -4.176737e-05, -0.009812801, -0.03677869, -0.0017592908, -0.0074769864, -0.049355242, 0.0027726663, -0.00534028, 0.010318004, 0.024582524, 0.005592881, -0.015976263, 0.021753393, 0.0058751996, 0.002300154, -0.005393772, 0.040844075, 0.0027325472, -0.0013098099, -0.015857391, -0.025081782, 0.019768247, 0.0018276417, -0.03176233, -0.008570599, 0.0030609283, -0.026413137, -0.007239244, -0.00885589, -0.019910892, -0.026888622, -0.00068202236, 0.0550135, -0.015120392, -0.004044586, 0.026460685, -0.0020668695, -0.07978622, -0.038157593, 0.016879683, -0.027435428, 0.019340312, -0.017212521, 0.024677621, -0.0085884305, 0.037040204, -0.060434017, 0.0016359622, -0.036113013, 0.01304015, 0.010234794, 0.026151622, -0.047049142, 0.0074472683, -0.016855909, 0.016546844, -0.011916818, -0.043435466, -0.0078038815, 0.009432415, -0.019043135, -0.025081782, 0.002946515, 0.012059463, 0.0009628551, 0.0018529018, 0.024035716, -0.025200652, 0.028505268, -0.020029765, 0.019245215, -0.0085052205, 0.0014160508, -0.008950987, -0.015049069, 0.024011943, 0.007946527, -0.011013399, -0.03131062, 0.017545361, -0.014514149, 0.012267488, 0.0061812922, 0.01086481, -0.021943588, 0.0015824703, 0.006288276, -0.013789036, -0.006793478, -0.00088113133, 0.011560205, 0.016451746, 0.009099576, -0.0051322556, -0.009628551, 0.026151622, 0.010852923, -0.0016731094, 0.041176915, 0.013777149, 0.015370021, 0.023358153, 0.017010441, 0.034639012, 0.011191705, -0.024368556, -0.05273118, -0.0025557266, 0.0007396005, 0.006912349, -0.0004877426, -0.00030925035, 0.0015089188, 0.012517117, 0.064998664, 0.013598843, -0.0065616793, 0.027910912, 0.002735519, -0.032118946, 0.008760793, 0.03238046, 0.017913861, -0.011898988, -0.016558731, 0.013254117, 0.068612345, 0.019578055, 0.02054091, -0.0048945136, -0.024915362, 0.0072035827, 0.0013855902, 0.0018543877, 0.032499332, 0.043720756, -0.033592947, 0.044624176, 0.03632698, -0.060481567, -0.030074364, -0.027411653, 0.00048959994, -0.000568352, 0.003607735, -0.015405682, -0.0046448843, 0.029955493, -0.0014844016, 0.019506732, 0.022062458, -0.007280849, 0.022062458, 0.062288407, 0.024261571, 0.032594427, 0.00049220026, -0.017176861, 0.0052332957, -0.008398237, -0.0047607836, -0.032974817, -0.016618166, 0.009794971, 0.046716306, 0.018377457, -0.0006861086, -0.012576552, -0.007560196, 0.0041129366, -0.02824375, -0.018674634, 0.0058722277, -0.008748906, 0.040891625, -0.02320362, 0.011221423, 0.0032838115, -0.009438357, -0.010169415, -0.014668682, -0.037991174, -0.026151622, 0.017913861, -0.04398227, -0.026104072, 0.0064428085, 0.03932253, 0.055251244, -0.012683536, 0.02501046, 0.017890086, 0.01901936, -0.0052808444, -0.02529575, -0.015025295, 0.015227376, 0.014050553, 0.021741506, 0.024891589, 0.0009866293, 0.014728118, -0.009485906, -0.014621134, 0.015239263, -0.048523143, 0.00475484, -0.023393814, 0.005313534, 0.036017913, 0.03801495, 0.015786069, -0.016499296, 0.0064428085, 0.037159074, 0.010953963, 0.016879683, -0.008475503, 0.044172466, 0.009913842, 0.012326923, -0.008742962, 0.013396762, 0.0026864847, -0.007637462, -0.026436912, 0.01582173, 0.004597336, 0.019613715, 0.008416067, -0.0036404245, -0.010490366, -0.025604814, 0.033521622, 0.0074710427, -0.013491859, -0.015274924, 0.001137447, 0.055774275, 0.027696945, -0.02529575, -0.02631804, 0.038110044, 0.012992601, 0.016653828, -0.00091902143, 0.016737038, 0.013479972, -0.0153105855, -0.0056909495, 0.019708812, -0.008517107, 0.006668663, 0.036350753, -0.038609304, 0.017866312, 0.0122793745, -0.037824754, -0.043673206, -0.0008514136, -0.020362603, -0.002035666, 0.03078759, -0.0088974945, 0.025414621, -0.008398237, -0.009652326, 0.0029658317, -0.015976263, 0.012398246, 0.033759367, -0.013598843, 0.006419034, 0.018745957, 0.011821722, 0.031381946, 0.0182467, 0.008784567, -0.0051411707, -0.0016315045, -0.015239263, -0.008267478, 0.010561689, 0.009783084, 0.014205085, 0.013943569, -0.038110044, -0.01561965, 0.065188855, -0.00067607884, -0.02139678, 0.017438376, 0.00739972, 0.015512666, 0.02824375, 0.01340865, 0.007815769, 0.012148617, 0.037991174, 0.033640493, -0.02693617, -0.004742953, -0.002839531, -0.016404198, -0.015536441, 0.015524553, 0.016844021, -0.04120069, 0.03525714, 0.0011344751, 0.0023120411, 0.008065398, -0.021432443, -0.00022046856, -0.012683536, 0.014038665, -0.017961409, -0.02841017, -0.04467172, 0.01279052, -0.034401268, 0.01799707, -0.007702841, -0.035328463, 0.01406244, 0.007185752, -0.029004525, -0.033141237, 0.004175344, 0.005548304, -0.030502299, -0.029242268, 0.017973296, 0.0026805412, 0.0025052065, -0.001057952, -0.02603275, 0.0067697037, -0.0065913973, 0.01980391, -0.03192875, 0.0048945136, 0.006430921, -0.002338787, -0.049925823, -0.023322491, 0.008731076, -0.02705504, -0.0032065453, 0.0078870915, 0.024118926, 0.03176233, 0.0028855936, 0.009551285, -0.03209517, 0.034662783, -0.014644908, 0.014086214, 0.030098138, -0.006104026, -0.0009286797, -0.0023046115, -0.019708812, 0.03887082, -0.0010193188, 0.019792022, 0.027031267, 0.015857391, 0.016665714, -0.002906396, -0.013349214, -0.037325494, -0.024487427, -0.00073477137, -0.02852904, 0.0054086307, -0.028029783, 0.023548346, -0.008107003, -0.026222944, 0.0060861954, 0.007173865, -0.0046983766, -0.015370021, -0.021052055, 0.027934685, 0.0109599065, 0.03841911, -0.02869546, -0.011031229, -0.008291253, -0.010401213, 0.033378977, -0.0394414, 0.008540882, -0.033474077, -0.03932253, -0.00021563943, 0.017438376, -0.019863345, -0.019138232, 0.04153353, 0.0050223, -0.044077367, 0.019982215, 0.014882649, 0.0139079075, 0.004282328, 0.01561965, -0.03226159, -0.005919776, -0.0012578038, 0.0074888733, -0.016166456, -0.007732559, 0.031025331, -0.005919776, -0.009670156, -0.047976337, -0.018115941, 0.032118946, 0.00041827734, 0.028885653, 0.0046062516, 0.0018142686, -0.05881737, 0.0063536554, 0.015595876, 0.0021263051, -0.019221442, -0.006484413, -0.036636043, 0.012683536, -0.008273422, -0.03801495, -0.027078815, -0.001560182, -0.016154569, -0.0012258572, 4.3787248e-05, -0.024962911, 0.05182776, -0.06290653, 0.028909428, 0.016487408, 0.010638955, 0.023358153, -0.03849043, 0.053777244, -0.013337326, 0.018175377, 0.0021738536, -0.035447333, -0.0140743265, 0.0068231956, 0.006246671, -0.0038395335, 0.01088264, -0.0065735667, 0.0029480008, -0.0031619687, 0.026175395, 0.013468085, 0.015381908, -0.016844021, 0.014502263, 0.02914717, -0.028338848, -0.0011211021, 0.010068374, 0.021789055, -0.00041827734, -0.022537943, 0.034377493, -0.040939175, -0.05178021, 0.019982215, 0.008023793, 0.031144204, 0.036136784, 0.034068428, 0.012142673, -0.0040624165, -0.047738597, -0.004056473, 0.033664268, 0.0006240728, 0.02841017, -0.015417569, -0.044624176, 0.0020401236, 0.009265995, -0.00039710346, -0.028600363, 0.01565531, 0.0035601866, -0.0077206716, -0.014573585, -0.01992278, 0.008683527, 0.0006578767, 0.026460685, 0.014133763, 0.0047607836, 0.043792076, -0.0016478493, -0.058056597, 0.024011943, -0.02017241, 0.036992658, 0.0059851552, 0.00785143, -0.018793507, 0.0055572195, -0.00977714, -0.021824716, -0.027078815, 0.003898969, 0.029527558, 0.016784586, -0.03169101, 0.011346238, -0.0078038815, -0.0047102636, 0.005578022, -0.030217009, 0.01975636, 0.030407203, 0.00904014, 0.025034234, 0.03841911, 0.04319772, 0.011465108, 0.016523069, 0.0042526103, -0.028671686, -0.01104906, -0.0061396873, -0.03131062, 0.008820228, 0.01664194, 0.0052273525, -0.04467172, -0.019958442, -0.031286847, 0.0073402845, 0.0075126477, 0.0069064056, 0.0023328434, -0.018175377, 0.02054091, 0.022110006, -0.0036790576, -0.0075780265, 0.02094507, 0.018044619, 0.010115922, -0.005821707, 0.0017429461, 0.011845496, 0.017699892, 0.016083246, -0.0123625845, 0.030644944, -0.0061218566, -0.030977784, 0.0012496314, 0.016511183, 0.01997033, -0.034092203, -0.021135265, -0.033806913, 0.038157593, -0.043340366, -0.007738502, -0.007637462, 0.0012711769, -0.007732559, -0.03689756, -0.008885607, 0.0074888733, -0.011898988, 0.029242268, 0.00053974864, 0.036754914, 0.047239337, 0.0015958432, 0.014466601, -0.0046151667, -0.016844021, -0.011958423, -0.02857659, 0.028172428, 0.004032699, 0.0014019349, 0.030098138, -0.013955456, -0.014335843, -0.033592947, -0.014930198, -0.015667198, 0.04307885, -0.012291262, 0.00024405702, -0.007334341, 0.0018945066, -0.025200652, 0.017533474, -0.007084712, 0.012303149, 0.01463302, -0.02054091, 0.017509699, -0.013563181, -0.030740041, 0.0048945136, 0.018282361, 0.012255601, -0.024653846, 0.01242202, 0.02213378, 0.017176861, 0.0068113087, 0.008445784, -0.023167958, 0.0010668673, 0.023084749, 0.008374462, 0.03711153, 0.017723667, 0.035352238, 0.0018855913, 0.025771234, 0.01771178, 0.02049336, -0.028743008, 0.0143833915, 0.03147704, 0.016867796, 0.029622653, -0.017830651, -0.02546217, -0.029575106, -0.015132279, -0.011964367, 0.031786107, -0.02000599, -0.03841911, -0.01160181, 0.0018945066, -0.009117406, -0.00256167, 0.037325494, -0.013955456, -0.0051857475, -0.008725132, 0.02139678, -0.00055163575, -0.035613753, -0.013670165, -0.008784567, -0.010222906, 0.03316501, -0.011631528, -0.0146092465, 0.020386377, -0.0032006018, -0.021610748, 0.009057971, -0.0051322556, -0.006609228, 0.029052073, -0.001055723, 0.019494845, -0.012861842, 0.020231845, 0.0025928738, -0.030359654, 0.017806876, -0.0009182785, -0.024154589, 0.008624092, -0.0023967365, -0.034377493, -0.029789073, -0.014811327, 0.0505915, 0.0073937764, 0.0041694003, 0.023952506, 0.023881184, 0.022264538, 0.034757882, -0.03775343, 0.013884134, -0.005482925, -0.006288276, -0.014704343, -0.017164974, -0.015227376, -0.012612213, 0.0014041638, -0.018139714, 0.0035245253, -0.015679086, 0.017985184, 0.034377493, 0.02586633, 0.006656776, 0.01043093, 0.017307619, 0.059530597, -0.007358115, 0.02909962, 0.027696945, 0.02660333, 0.006341768, 0.008469559, 0.011055003, 0.02139678, 0.021693958, 0.0057384977, 0.010359608, 0.005099566, 0.012552778, 0.039132334, -0.0312393, -0.027031267, -0.025176879, -0.019055022, -0.001771178, 0.028196203, -0.0021188757, -0.012481456, -0.006698381, -0.025509717, 0.0011300175, 0.018686522, 0.011447278, -0.01014564, -0.01242202, 0.002934628, 0.025081782, 0.003818731, -0.021800943, -0.02254983, -0.007791994, -0.035209592, -0.0033016421, 0.028838105, -0.007988132, -0.010805374, 0.017450264, 0.014181311, -0.04374453, -0.008309083, 0.0002815757, -0.0015898996, 0.030930236, 0.013123359, 0.019411635, -0.02783959, 0.0085884305, -0.0033462187, -0.030193236, 0.035376012, 0.008368519, 0.017937634, -0.022169443, -0.0045408723, -0.009396753, 0.028743008, -0.004546816, -0.02574746, 0.027578073, 0.0394414], "content_hash": "928d61d5538838576b0ce31317012e35"}, {"content": "GPT-style models) Most LLMs you'll encounter in this dashboard are **decoder-only** models. This means they generate text left-to-right, predicting one token at a time based on everything that came before it. Each token can only \"see\" the tokens to its left -- it cannot look ahead. ## The Residual Stream\n\nThere is an important concept called the **residual stream** (or \"residual connection\"). Think of it as a conveyor belt running through all the layers. Each layer reads from this stream, does some processing, and adds its result back. This means information from early layers is preserved and can be used by later layers.\n\n## How This Connects to the Dashboard\n\nThe dashboard's 5-stage pipeline follows the exact path data takes through a Transformer: Tokenization, Embedding, Attention, MLP, and Output. When you expand each stage, you're seeing what happens at that point in the architecture.", "source_file": "transformer_architecture.md", "chunk_index": 1, "embedding": [0.004264763, 0.035261303, 0.013565947, -0.0026626983, 0.022495592, -0.007373626, 0.012384643, 0.047582433, 0.001895803, 0.05467026, -0.013438926, -0.0041409167, -0.033559207, -0.04153619, -0.031831708, -0.043873392, 0.0026214162, 0.001240846, 0.0011352589, 0.062850475, -0.0012368766, -0.020475943, 0.011933715, 0.046845708, -0.01190196, 0.0042742896, 0.0042330073, 0.06442555, -0.019193022, 0.0151410205, 0.0033660824, -0.032314394, -0.030891746, -0.009380573, 0.029240461, -0.0026245916, 0.015814237, 0.05223144, -0.01987894, 0.04374637, 0.022152634, 0.016322324, -0.076416425, 0.036506116, 0.0022435258, 0.05807445, -0.012054386, -0.0151410205, -0.036658544, 0.06727084, -0.015306149, 0.022863956, -0.014074036, 0.015687214, -0.036048837, -0.037166633, -0.038259022, -0.014861572, -0.0208062, -0.024769286, 0.010206216, -0.0023356166, 0.016601773, 0.0032136561, 0.016868519, -0.001247197, -0.033965677, 0.026623808, -0.030561488, 0.030866342, 0.039427623, -0.009170987, -0.00999663, 0.0036614086, 0.01872304, -0.0011193812, 0.0049729105, 0.032314394, -0.011406574, -0.03300031, 0.024019856, -0.023422854, 0.008135758, -0.037319057, -0.00066805625, -0.00999663, -0.0027452626, -0.008250077, -0.011304956, -0.05421298, -0.0051221615, -0.023740409, -0.05223144, 0.0116479155, 0.017097158, 0.029748548, 0.023372045, -0.035845604, -0.017236883, 0.01398512, 0.006655952, -0.008948699, -0.0011955944, -0.04783648, 0.017287692, -0.060513273, -0.031247407, -0.016843114, -0.06498445, -0.047480818, -0.070370175, -0.022495592, 0.001205121, 0.009190041, 0.06722003, -0.010002981, -0.023270426, -0.008383451, -0.011755885, -0.023943644, -0.00020383058, 0.027843218, 0.010009333, -0.0017195601, -0.024591455, 0.022457486, 0.0036677597, -0.02572195, 0.0093043605, -0.026852448, -0.013667565, 0.0151410205, 0.015979365, -0.04514361, -0.013095967, -0.002361021, -0.033355974, 0.00548735, -0.014721848, 0.005506403, 0.035210494, -0.026217338, 0.01120969, -0.02103484, -0.026420573, 0.014925083, -0.0323652, 0.044051223, -0.065746576, 0.015725322, 0.010447558, 0.0046839355, -0.022978276, -0.02443903, -0.033559207, -0.0007363306, -0.012505314, 0.020209197, 0.032492224, -0.030536084, -0.029240461, 0.016220707, 0.04501659, 0.022178037, -0.06406989, 0.04270479, -0.02178427, 0.012695847, -0.047404602, -0.04224751, -0.020171091, 0.00096616097, -0.039707072, -0.0007494297, 0.0009534588, -0.04593115, -0.08815326, -0.030967958, 0.03155226, -0.057464745, -0.033737037, 0.007900767, -0.038513064, -0.022813149, -0.05975114, 0.019434363, -0.012117897, 0.06915077, -0.0072084973, 0.012727602, -0.02670002, 0.065289296, 0.054416217, 0.034397554, -0.006252657, 0.0054524187, 0.0502753, 0.0075387545, 0.03132362, 0.0027182703, 0.031349026, 0.040596224, 0.02346096, -0.068642676, -0.013642161, 0.07336789, 0.002481692, 0.0034994555, -0.025213864, -0.026598403, 0.007888065, -0.02155563, 0.016589072, -0.008599388, -0.049563978, -0.011419276, -0.04986883, 0.016703391, 0.037928764, 0.012124248, 0.006884591, 0.031984136, 0.008764517, 0.012092493, -0.06340937, 0.026827043, 0.015636407, -0.012505314, -0.0011050912, 0.0040075434, 0.017147968, -0.008986806, -0.0015028288, -0.006865538, 0.005026895, -0.062139153, -0.008186567, -0.027131895, 0.02508684, 0.013261095, -0.030104209, 0.018888168, 0.017465523, 0.002959612, 0.038360637, 0.020933222, -0.02495982, -0.018176846, 0.032212775, 0.033330567, 0.024756584, 0.0019275586, -0.06986209, 0.029850166, -0.026877852, 0.024057964, -0.038411446, 0.0042552366, -0.024896309, -0.0069354004, 0.04374637, 0.020171091, -0.03345759, 0.032720864, 0.0023435554, -0.014671039, 0.010892135, -0.04488957, -0.029291268, -0.038208213, 0.006332046, -0.046261407, -0.02189859, 0.012327483, -0.031196598, -0.037065014, 0.033355974, -0.0054937005, 0.022571806, 0.05385732, -0.0012797464, -0.0520028, 0.055737246, 0.04415284, -0.031425238, 0.013743779, -0.0055445093, -0.004344152, -0.01600477, -0.018087931, 0.017262287, 0.010060141, 0.035337515, -0.022152634, 0.015331553, -0.001973604, 0.0065670363, 0.017122563, -0.0010908013, -0.019218426, 0.018799253, -0.030434467, -0.036455307, 0.009990279, -0.013070562, 0.025226565, 0.016131792, 0.0111080725, 0.00013297614, -0.012086142, -0.02184778, -0.032492224, 0.06742327, 0.035439134, 0.011768587, -0.003327976, 0.036429904, -0.037623912, 0.012994349, -0.035896413, -0.008472366, -0.021695355, -0.027716197, -0.036912587, -0.0103268875, 0.0083008865, -0.018075228, -0.010434856, -0.022000207, -0.027081087, 0.043365303, 0.003327976, -0.019612193, 0.04519442, -0.046540853, -0.027919432, -0.0039853146, 0.016970137, -0.008097651, 0.020132985, -0.058582537, -0.020577561, -0.036785565, -0.004372732, -0.016195303, 0.029316673, -0.032238178, 0.027614579, 0.039376814, -0.034651596, -0.0073101153, -0.023257725, -0.017948207, -0.023778515, 0.039605454, 0.030231232, 0.0108667305, 0.023118, 0.02756377, -0.0062018484, -0.0572107, -0.030409062, 0.0015496681, 0.022203442, -0.05159633, 0.030155018, -0.027589174, -0.020044068, 0.009742586, -0.022787744, 0.052485485, 0.0039789635, 0.007265657, 0.03322895, -0.0543146, -0.025353588, -0.0026023628, 0.0032581137, -0.04524523, 0.018164143, 0.00019569324, 0.012327483, -0.0043028696, -0.0151410205, 0.024375519, 0.054619454, -0.029773952, -0.025671143, -0.02362609, 0.01195912, 0.001846582, -0.029316673, 0.013832694, 0.013375415, -0.0055953185, -0.013477032, 0.026623808, 0.0108857835, 0.03213656, 0.033432186, -0.008415206, -0.007551457, -0.035337515, -0.049691, 0.022393975, 0.015687214, 0.027817814, 0.0012202049, 0.04697273, -0.015191829, 0.057007466, 0.0036995152, 0.009660021, -0.032111157, -0.011216041, 0.038335234, 0.025823569, 0.008402504, 0.012225865, 0.065390915, -0.019751918, 0.014074036, -0.056651805, -0.0034581735, -0.023016384, 0.004991964, 0.055025924, -0.030078806, -0.0040138946, -0.033533804, 0.02391824, 0.031018768, 0.04085027, -0.039275195, 0.0006208199, -0.029113438, 0.017122563, 0.0043822583, -0.04709975, -0.020031366, 0.054568645, -0.0028786357, -0.023245022, -0.030942556, -0.061986726, -0.012822869, -0.04120593, -0.04638843, -0.029545313, -0.04293343, 0.023016384, 0.05159633, 0.033686228, 0.021098351, -0.041002695, -0.07153878, 0.0681854, 0.018799253, 0.035820197, -0.002291159, 0.014074036, 0.024362816, -0.019853536, 0.0038233616, -0.04374637, 0.0066114943, 0.0213778, -0.025925186, 0.036429904, 0.013819992, -0.020209197, -0.0340927, 0.012162355, -0.006624196, -0.014251866, 0.0015345843, 0.016512858, -0.036582332, -0.004328274, -0.034448363, 0.029189652, 0.009742586, -0.026344359, -0.010790518, -0.0017973611, 0.11838449, -0.04524523, -0.01958679, -0.018405486, -0.014277271, 0.0013424635, -0.029621527, -0.015077509, -0.013134073, 0.016131792, -0.019053297, -0.07184363, -0.07153878, 0.0056080204, -0.004179023, 0.028224284, 0.017846588, -0.028097263, 0.04257777, 0.042552363, -0.0012995936, 0.017986313, 0.02207642, 0.002700805, 0.012359239, -0.007957927, 0.01219411, -0.0085422285, 0.017109862, -0.013095967, 0.0068972935, -0.01259423, 0.013908907, -0.0007406969, 0.026674615, 0.005246008, 0.0055603874, -0.018189548, -0.018126037, -0.009348818, -0.006011315, -0.020653775, -0.091252595, -0.03386406, 0.009793395, 0.013184882, -0.012860975, -0.014264569, 0.0072720083, -0.05032611, -0.0059446287, -0.0048966976, -0.05426379, -0.02466767, -0.005512754, 0.010358643, 0.037395272, 0.01254342, 0.022927467, -0.012619634, 0.0065035257, 0.012067088, -0.054822687, 0.0057413937, 0.04577872, 0.026217338, -0.041993465, 0.028275093, -0.013146775, -0.034168914, -0.018303867, -0.015052104, 0.012371941, -0.008650197, -0.033533804, -0.0040107192, 0.026039505, 0.021022137, -0.035591558, -0.02502333, 0.033127334, 0.011686022, -0.0028659333, 0.012454505, -0.014582124, -0.020005962, -0.0040932833, 0.007983332, 0.021695355, -0.021771567, 0.005858889, -0.00033521894, 0.028021049, 0.0038043084, 0.00012563268, -0.028402114, -0.05512754, -0.030993363, -0.011317658, -0.028681563, 0.04872563, -0.010339589, -0.006176444, -0.02045054, 0.019777322, 0.046591662, -0.015801534, -0.014950487, 0.034473766, -0.034905642, 0.004664882, 0.035235897, -0.008681952, -0.035210494, -0.012124248, 0.014175653, -0.011406574, -0.031603068, 0.0014829816, 0.021415906, -0.001649698, -0.028808586, 0.039859496, -0.018468997, -0.030536084, -0.018087931, -0.02160644, -0.013565947, -0.028986417, -0.055889674, 0.02299098, -0.053145997, 0.009545702, 0.05050394, 0.010854028, -0.004096459, -0.018977083, 0.0046013715, -0.032974906, 0.003877346, 0.009005859, -0.013210286, 0.007119582, -0.05126607, 0.001995833, -0.04067244, -0.001402799, 0.06437474, 0.008885188, -0.002029176, 0.016258813, -0.001846582, 0.0037280952, 0.021644546, -0.025353588, -0.003045352, -8.484672e-05, -0.006408259, -0.0067639207, 0.012225865, 0.010441207, 0.017681459, 0.027081087, -0.0219494, -0.009945821, -0.024261199, -0.034524575, 0.068642676, 0.011914662, 0.009507596, -0.007824554, -0.0008256429, 0.0041504432, 0.0066368985, 0.015445873, -0.023435555, -0.011832098, -0.029748548, -0.026979469, -0.026344359, 0.050478537, 0.015382362, 0.008275482, 0.029951783, 0.012702198, -0.057820406, 0.02629355, 0.008993156, -0.008186567, 0.018392783, -0.02287666, -0.036455307, -0.06594981, -0.015814237, 0.0044235406, -0.0011797167, 0.0035375622, -0.026674615, 0.00048030188, -0.0072720083, -0.01357865, 0.022914765, -0.0467949, 0.01144468, 0.0045061046, -0.014823466, -0.011533596, 0.012492612, -0.026979469, -0.02670002, -0.0048046065, -0.0071386355, 0.07580672, 0.012702198, -0.031120386, 0.0030437643, -0.033991084, -0.036277477, -0.0008200857, 0.0036487065, -0.002248289, 0.017059052, 0.029697739, -0.007646723, -0.04085027, -0.036506116, 0.021644546, 0.018037122, -0.014175653, -0.027538365, 0.024693074, -0.01213695, -0.012340185, -0.002087924, -0.02531548, -0.050478537, 0.02692866, -0.018240357, -0.011133477, 0.010822273, -0.043720964, -0.037522294, -0.032924097, -0.037598506, 0.004299694, 0.006662303, 0.03980869, -0.007005262, -0.04125674, -0.013007051, 0.0022990978, 0.004582318, -0.014759954, -0.02316881, -0.001486951, -0.027665388, -0.0016592246, -0.04610898, 0.035489943, -0.006309817, 0.028961012, 0.0011328773, 0.030358253, 0.0003088222, -0.06442555, -0.0038614683, 0.010193514, 0.0001751514, 0.007983332, -0.015522086, 0.010218918, 0.02733513, -0.004512456, 0.016436644, -0.026090315, 0.02560763, 0.018786551, 0.00543019, 0.021619141, -0.003721744, -0.028961012, 0.022381274, 0.017033648, 0.032161966, 0.01357865, 0.018684933, -0.02184778, 0.03213656, 0.00034514253, 0.027792409, -0.024997925, 0.028300498, 0.016919328, -0.017020946, 0.008834379, -0.0052047255, 0.0039821393, 0.0048331865, -0.015992068, -0.001064603, 0.0067512183, -0.038741704, 0.009190041, -0.0026626983, -0.031425238, -0.024642264, 0.008516824, 0.018227654, -0.01409944, 0.0010177636, -0.0045315092, 0.017389309, -0.115335956, -0.041409165, -0.0038233616, -0.015153723, 0.015522086, 0.0010257025, 0.031018768, -0.016970137, 0.015928557, -0.034473766, -0.017719567, -0.033737037, 0.006649601, -0.01577613, 0.017808482, -0.050122876, 0.0021958924, -0.025277374, 0.016868519, -0.007386328, -0.050757986, -0.02391824, 0.011133477, -0.0037249196, -0.06274886, -0.024743881, -0.014289973, 0.0072021466, 0.026623808, 0.055229157, -0.0022927467, 0.01323569, -0.011755885, 0.0011908311, 0.0020180617, -0.0046426533, -0.0029564367, -0.041993465, 0.028148072, 0.017186074, -0.015280744, -0.032111157, -0.009183689, -0.009158285, 0.027309727, -0.015801534, -0.012937189, -0.020247303, -0.0044743493, -0.0016957434, -0.027284322, -0.014340782, 0.00087803946, 0.018494401, 0.017706864, 0.02393094, 0.016487453, -0.0069481023, 0.011044561, 0.0052110767, -0.015903153, 0.015242638, 0.015090211, -0.0146583365, 0.017783077, 0.010161758, 0.027411344, 0.033889465, -0.039249793, -0.039249793, -0.012187759, -0.018596018, 0.018634126, 0.006309817, -0.007862661, 0.020310815, 0.009850555, 0.049386147, -0.0116479155, 0.015839642, 0.012111546, 0.001057458, -0.057464745, 0.0065733874, 0.02531548, 0.013642161, -0.00067281956, -0.016893923, -0.0014401117, 0.029977188, 0.038817916, -0.0008335817, -0.026877852, -0.015445873, -0.0027468503, -2.3134375e-05, -0.011914662, 0.011686022, 0.059192244, -0.020196496, 0.039427623, 0.009075721, -0.06290129, -0.022559104, -0.023041787, 0.0001646324, 0.020018665, -0.010117301, -0.01086038, -0.03363542, -0.009374222, -0.029240461, 0.011050913, 0.018481698, -0.021644546, 0.007875362, 0.068744294, 0.03282248, 0.0467949, -0.023842026, -0.0070179645, -0.0045156316, -0.004118688, 0.011533596, -0.026141124, 0.007945225, 0.016487453, 0.034880236, 0.011133477, -0.005420663, -0.015433171, 0.003791606, -0.003059642, -0.028021049, -0.023676896, 0.0146583365, -0.01195912, 0.049386147, -0.044813354, 0.013565947, -0.012645038, 0.008701006, 0.029545313, -0.02074269, -0.015356958, -0.007869012, -0.000352486, -0.027639983, -0.007767394, 0.028808586, 0.034448363, 0.046718683, -0.017643353, 0.0064654187, -0.006033544, 0.018646827, -0.007125933, -0.06417151, -0.02386743, 0.00814846, 0.041917253, 0.044508502, -0.012587878, -0.020437837, 0.04062163, -0.01958679, -0.0093234135, 0.007957927, -0.033914868, 0.016360432, -0.044279862, 0.007989682, 0.024997925, 0.007430786, 0.031069577, 0.0025642563, -0.0023594333, 0.035489943, 0.008275482, 0.011025508, -0.024997925, 0.056905847, -0.007761043, 0.037928764, 0.010263376, -0.012435452, 0.0013321431, 0.0033629069, -0.048624016, 0.022114526, 0.010314185, 0.016779603, 0.0146583365, 0.020196496, -0.035972625, -0.02629355, 0.049487762, 0.017008243, -0.016868519, -0.0035820198, 3.0812753e-05, 0.021809675, 0.017656056, -0.004547387, 0.003108863, 0.01201628, 0.022038314, 0.0040488257, -0.006344748, -0.0018719864, -0.0032342973, 0.00064900296, -0.008332642, 0.023041787, -0.010314185, -0.008535877, 0.04021516, -0.0054746475, 0.028656159, -0.0068210806, -0.029519908, -0.022139931, -0.0052047255, -0.008015087, 0.0025706075, 0.034931045, -0.0143788885, 0.0076657766, -0.009151934, 0.012003577, 0.0066940584, 0.0048808195, 0.0055508604, 0.004118688, -0.022139931, -0.0021355571, 0.0055000517, 0.008167514, 0.03975788, 0.018900871, 0.008472366, 0.009545702, -0.030027997, -0.029265864, 0.008701006, -0.013083264, 0.012670442, 0.0009891837, -0.010841326, -0.01143833, 0.016855817, 0.060310036, 0.02316881, -0.0067512183, -0.0031961906, -0.0018227655, 0.015064807, 0.02507414, 0.0062875883, 0.019332746, 0.009526649, 0.05426379, 0.029773952, -0.029672334, 0.01335001, 0.0011566939, -0.005166619, 0.007856309, 0.011952768, 0.015674513, -0.008808974, 0.022863956, -0.0073990305, -0.0070878267, 0.01259423, -0.0068274317, 0.014366186, -0.014010524, 0.012416399, -0.018811956, -0.017795779, -0.05350166, 0.001001092, -0.031349026, 0.019891642, -0.0069481023, -0.029265864, 0.004934804, 0.017719567, -0.020171091, 0.0062081995, 0.013845396, 0.028706968, -0.007291062, -0.008326291, 0.0029310323, 0.018303867, -0.013108669, 0.020132985, -0.058938198, -0.009399626, -0.0047665, 8.2415434e-05, -0.032390606, 0.0013337308, -0.011412925, 0.013794587, -0.02404526, -0.005928751, -0.0018640476, -0.017821183, -0.01103821, 0.016360432, 0.022279656, 0.02466767, -0.0034772267, 0.018329272, -0.030790128, 0.012333835, -0.038843323, -0.0085422285, 0.012803815, -0.010466611, -0.0025023331, 0.019561386, 0.0027722546, 0.024997925, 0.0018021243, 0.009463138, 0.020387027, 0.02606491, -0.00066091126, -0.0029643755, -0.011736831, -0.02339745, -0.021885887, -0.013972418, -0.041002695, -0.018583316, -0.039605454, 0.045448463, 0.012314782, -0.025239268, 0.003963086, -0.0009828326, 0.027462153, 0.031831708, -0.027792409, 0.05390813, -0.0033629069, 0.04412744, -0.038004976, -0.015026701, -0.00641461, -0.011425627, -0.00077800965, -0.008948699, -0.006373328, -0.022292357, -0.01167332, 0.016550964, 0.035591558, -0.013947014, -0.01999326, 0.02652219, -0.0051570926, -0.02056486, -0.001275777, 0.019053297, 0.011317658, 0.015318851, 0.0083008865, -0.006373328, -0.012276675, -0.018621422, 0.004668058, -0.013400819, -0.01213695, 0.020298112, 0.014201057, -0.00803414, -0.042095084, -0.0012003577, 0.015458575, -0.0036677597, 0.0260014, 0.025378993, 0.023384746, -0.04560089, -0.01641124, 0.027131895, -0.009374222, -0.016881222, -0.007945225, -0.021568332, -0.0031580841, 0.01652556, -0.017097158, -0.029951783, -0.031298216, -0.016309623, -0.005744569, 0.003963086, -0.012740305, 0.039376814, -0.05807445, -0.005280939, 0.025163054, -0.0020275884, 0.005134864, -0.038843323, 0.011546298, 0.0098442035, 0.003693164, 0.0072529553, -0.044635523, -0.010390398, 0.014340782, -0.0111080725, 0.023994451, 0.007919821, -4.5921424e-05, -0.017109862, 0.0006140719, 0.028529137, 0.011622511, 0.014556719, -0.0015853931, 0.008135758, 0.028173475, -0.029392887, -0.0061097573, -0.030561488, 0.011057263, 0.0052999924, 0.004318747, 0.038741704, -0.021822376, -0.034041893, 0.0057922024, 0.018596018, 0.01525534, 0.020425135, 0.019548682, 0.014048631, -0.017351203, -0.038995747, -0.02443903, 0.02039973, 0.008199269, 0.01196547, -0.026420573, -0.020602966, 0.015509384, -0.010523771, 0.0005970033, -0.05680423, 0.025455205, -0.011940067, 0.005477823, -0.011196988, -0.008065896, -0.00470934, 0.01254342, 0.021009436, 0.024756584, 0.017529033, 0.055483203, -0.011139828, -0.026623808, 0.033737037, 0.002184778, 0.031907924, -0.015903153, 0.022089122, -0.012467207, -0.01282922, 0.018227654, -0.014277271, -0.035007257, -0.021161862, 0.040723246, 0.0141375465, -0.028452924, -0.017478224, -0.012283025, 0.003135855, -0.024299305, -0.033889465, 0.04765865, 0.015877748, 0.0039313305, -0.005246008, -0.0015901565, 0.032949504, 0.023181511, 0.016157197, 0.010993753, -0.024235794, 0.014429697, -0.016208004, -0.018291166, 0.010593633, -0.012384643, 0.015039403, -0.036252074, 0.0025213864, -0.007983332, 0.005074528, 0.0064876475, 0.0011868617, 0.0016322325, -0.04743001, 0.023956345, 0.008662899, 0.014594826, -0.01566181, 0.018481698, 0.0123909945, -0.014925083, -0.028249688, -0.011158881, 0.008834379, 0.024781989, -0.0063193436, -0.025582228, 0.016042877, -0.004347327, -0.033533804, 0.013197584, -0.017554438, 0.015128318, 0.022317762, 0.0012995936, -0.0034232424, 0.023842026, -0.032263584, -0.00061010243, -0.0068401336, 0.00060732383, 0.009812448, -0.026674615, -0.02362609, 0.004207603, -0.012200462, 0.032543033, 0.0013242041, 0.033686228, 0.0404438, 0.004080581, 0.00021256335, -0.015725322, -0.024019856, -0.001247991, -0.023118, 0.032492224, -0.017147968, -0.024921713, 0.0093424665, -0.008218322, -0.034829427, -0.010580931, -0.019396257, -0.010257025, 0.048573207, -0.001987894, 0.0025150352, -0.0066432497, 0.013858099, -0.02652219, 0.021225372, -0.026496785, -0.0008161162, 0.019066, -0.003186664, 0.0088216765, 0.008675601, -0.02039973, 0.02520116, -0.025531419, 0.0013289675, -0.03993571, 0.005490525, 0.003518509, 0.018697636, 0.019383555, -0.007259306, -0.019294638, -0.00704972, 0.0014250278, 0.0024546997, 0.026725424, 0.026852448, 0.02670002, 0.020958627, 0.031450644, -0.0038170104, 0.0074879457, -0.047353793, 0.031526856, 0.013477032, 0.010231621, 0.0063987323, -0.030104209, -0.02507414, -0.0008327879, -0.016716093, 0.011819395, 0.015407766, -0.00635745, -0.013388117, 0.014785359, 0.016538262, -0.012606932, 0.028452924, 0.0154204685, 0.00652893, -0.007983332, -0.01335001, 0.029088033, 0.001353578, -0.029672334, -0.000641858, -0.016322324, -0.011876555, 0.05017368, -0.019891642, 0.009151934, 0.009520298, -0.0013027692, -0.0068464847, 0.021403205, -0.0076657766, -0.013451627, 0.012276675, -0.014874274, 0.010669846, 0.004490227, 0.018507103, -0.009228147, -0.028986417, 0.018265761, -0.0031580841, -0.023956345, 0.021758866, 0.004115512, -0.019624896, -0.038208213, 0.012448154, 0.043822583, -0.008770868, 0.012575177, 0.017071754, 0.016500155, 0.024921713, 0.042196702, -0.011832098, -0.0074498393, -0.006309817, -0.025391694, -0.000398333, 0.0029691388, -0.017033648, -0.0054682964, 0.02288936, -0.030256636, -0.0041536186, -0.026268145, 0.016563667, 0.03241601, 0.013642161, -0.004369556, -0.0039567347, 0.026674615, 0.04722677, -0.007297413, 0.014975891, 0.006373328, 0.024134176, -0.0046299514, 0.0151410205, 0.015331553, 0.017986313, 0.018075228, 0.008472366, 0.013477032, 0.017630652, 0.011000103, 0.029570717, -0.00050014904, -0.020412432, -0.0018370554, -0.034245126, -0.007634021, 0.03200954, 0.013095967, -0.025747355, -0.039148174, -0.009869608, 0.0116479155, -0.0011082668, 0.025163054, -0.025290076, -0.02149212, 0.020958627, 0.008065896, -0.00027607434, -0.0042266566, -0.041104313, -0.011832098, -0.030790128, -0.030866342, 0.035998028, -0.0032152438, -0.00057358365, 0.01167332, 0.008421557, -0.067016795, -0.032746267, 0.013743779, -0.005223779, 0.02039973, 0.025683844, 0.0018338799, -0.014010524, -0.010968348, -0.024642264, -0.0035820198, 0.038030382, 0.013095967, 0.027182704, 0.005928751, 0.0021466715, -0.006274886, 0.008478717, -0.021936696, -0.0074752434, 0.028884798, 0.010472963], "content_hash": "22be9f22e1d3793d7afd1facc577b6b5"}, {"content": "# Troubleshooting and FAQ\n\n## Common Issues\n\n### Model takes a long time to load\n\n**Why**: The first time you load a model, it must be downloaded from HuggingFace. GPT-2 (124M) is about 500MB; larger models are much bigger.\n\n**Fix**: Be patient on the first load. Subsequent loads should be faster because the model is cached locally. If loading is consistently slow, try a smaller model.\n\n### Ablating a head has no effect\n\n**Why**: Not every head is important for every input. Many attention heads are redundant -- the model has learned to distribute work across multiple heads, so removing one doesn't always change the output.\n\n**Fix**: This is actually an interesting finding! Try:\n- Ablating a head from a different category (Previous-Token heads often show more effect)\n- Using a different prompt (some prompts depend more on specific heads)\n- Ablating multiple heads simultaneously to see if their combined removal has an effect\n\n### Attribution takes too long\n\n**Why**: Integrated Gradients is computationally expensive because it runs the model multiple times (typically 50 steps) to build up the attribution scores.\n\n**Fix**: Switch to \"Simple Gradient\" for faster (though less accurate) results. Or use a shorter prompt -- fewer tokens means faster computation.\n\n### The model's prediction seems wrong or nonsensical\n\n**Why**: Small models like GPT-2 (124M) have limited knowledge and can produce incorrect facts, repetitive text, or non-sequiturs. The model was trained on data from before 2019 and has a limited understanding of the world.\n\n**Fix**: This is expected behavior for small models. The dashboard is designed for exploring *how* the model works, not for getting useful outputs. Try different prompts or a different model.\n\n### BertViz visualization is hard to read\n\n**Why**: With 12+ heads selected simultaneously, the attention lines overlap and become a dense mess.", "source_file": "troubleshooting_and_faq.md", "chunk_index": 0, "embedding": [-0.028126141, 0.018463913, 0.0643142, 0.0061068297, -0.053293224, 0.0021551298, -0.007276865, 0.022102347, 0.0058615, 0.050183196, 0.0011294616, -0.02862435, -0.041638162, -0.03321391, -0.013836612, 0.00029062168, -0.00027929878, 0.0068654655, -0.008756394, 0.05806395, 0.010318957, -0.012817549, 0.022223124, 0.03831677, 0.044567026, 0.03451227, 0.028397892, 0.014840578, 0.045321885, -0.027401473, 0.015248204, -0.019686788, -0.061204173, -0.06226098, -0.0108398115, 0.01782983, 0.010568062, 0.02229861, -0.0036761756, 0.05537664, -0.023038374, -0.004095124, -0.03182496, 0.029666059, -0.012734515, 0.018388428, 0.026676808, -0.003766759, -0.06322721, 0.051270194, -0.014297077, 0.06491809, -0.052266613, 0.002368378, -0.0030402048, -0.023778139, -0.019762276, -0.013104397, -0.004929246, -0.0115267355, 0.045684222, 0.013361049, -0.009533902, 0.02829221, 0.008960207, -0.028971586, -0.02551432, 0.00030383177, -0.029696254, -0.03004349, -0.008937561, -0.021287095, 0.027235404, -0.022540165, 0.00637858, -0.0025552064, 0.009035693, 0.044778388, 0.003764872, -0.0419703, 0.0096999705, -0.010266117, -9.394488e-05, -0.028941391, 0.0013266691, -0.023310125, -0.049851056, -0.032610018, -0.04486897, -0.06419343, 0.017965704, 0.030269949, -0.025227472, 0.007865657, 0.0042008045, -0.037471328, 0.01897722, -0.021105928, 0.003076061, 0.013066653, 0.014417855, -0.01576151, 0.0005118905, -0.001545579, 0.02791478, -0.022223124, 0.0067182677, 0.00010656522, -0.01793551, -0.0017399559, -0.07264788, 0.011979653, -0.03085874, -0.01810158, -0.04160797, -0.017256135, -0.06896415, 0.03282138, 0.041547578, -0.03303274, 0.022117443, 0.0140404245, 0.015399176, -0.0067975284, -0.0018446929, -0.006303094, 0.012470312, -0.02774871, -0.00806947, -0.020079317, 0.015489759, 0.027114626, 0.036988217, -0.024246152, -0.005552007, -0.03303274, -0.031583406, 0.01989815, -0.035840828, 0.009858492, -0.048009194, -0.02463868, -0.013232723, 0.014055522, -0.042000495, 0.030345434, 0.029892517, 0.018086482, -0.058305506, 0.026676808, 0.0029043297, 0.028639447, -0.012960972, -0.057127923, -0.012696771, -0.061355148, 0.025604904, 0.06727326, -0.04290633, -0.07910949, 0.016289912, 0.03004349, -0.035146352, -0.0057482705, -0.02578607, 0.09909822, -0.036021993, -0.029258434, -0.003066625, -0.09203272, 0.027975168, -0.033546045, 0.017799634, -0.05739967, 0.022389194, 0.023159152, -0.07766016, -0.032791186, 0.014765092, -0.048854638, -0.030873837, 0.00389886, -0.010205728, 0.013881904, -0.052749723, -0.035357714, -0.04399333, 0.020970054, -0.0027571318, 0.02213254, 0.015625633, 0.014976453, 0.056191895, -0.023446, -0.02164943, -0.03849794, 0.06962843, -0.0011870197, -0.0062238337, 0.03152302, 0.0046914644, 0.025604904, 0.05468217, -0.044385858, 0.0023098763, -0.017210843, -0.00466127, 0.03330449, -0.052477974, -0.014251786, 0.021468263, -0.036776856, 0.015278398, 0.021936277, -0.044959553, -0.014183848, -0.04973028, 0.014878321, 0.0072277994, 0.048190363, 0.011790938, 0.041124858, 0.01330066, 0.004940569, -0.018962122, 0.01935465, 0.043661192, -0.024487708, 0.027990267, 0.021302193, 0.031039907, -0.0029968002, 0.003532752, 0.019218775, 0.033244103, -0.045321885, 0.03892066, 0.013708286, -0.0046310755, 0.0006029457, 0.03735055, 0.023672458, 0.007948692, -0.01297607, 0.036927827, 0.0050839926, -0.051994864, -0.068662204, 0.0485225, 0.032700602, 0.038045023, -0.0218004, -0.018463913, -0.005948309, -0.009209311, -0.0108171655, -0.03411974, 0.008084567, -0.025876654, -0.016652245, -0.0045291693, 0.030556796, -0.058275312, -0.00039960482, -0.022494875, -0.0044423603, -0.03221749, 0.0019192355, -0.07222515, -0.051662724, 0.025484126, -0.019128192, 0.035418104, 0.011949458, -0.04655986, -0.011662611, -0.0043630996, 0.027899683, -0.03254963, -0.01210043, 0.0028326178, 0.009262151, 0.030662475, 0.0283526, -0.019943442, 0.010235922, -0.011617319, -0.014795287, -0.024759458, 0.0068163997, 0.016591856, -0.024789654, 0.0078732055, 0.02747696, 0.009224408, 0.004400843, 0.030134073, 0.009609387, -0.023672458, -0.031039907, -0.005804885, -0.05223642, -0.0028250692, 0.0053557428, 0.022238221, 0.024910431, -0.009148922, 0.04351022, 0.014704702, 0.003644094, -0.03297235, -0.024427319, -0.007182508, 0.004732982, -0.023083666, -0.024246152, 0.0063559343, -0.0010105709, 0.024683973, -0.008235539, -0.04018883, -0.07144009, -0.025635097, 0.043570608, 0.026737196, -0.003068512, -0.005986052, 0.019445233, -0.011707903, -0.010077402, 0.025151987, -0.0022287287, -0.025151987, 0.01733162, -0.02862435, -0.002109838, -0.025831362, 0.030179365, -0.002859038, 0.043540414, -0.07729782, -0.0033629083, -0.011322923, -0.0019739629, -0.06654859, 0.031070102, 0.027129725, 0.004340454, 0.018222358, 0.0030590764, -0.021694722, 0.01749769, -0.029122557, -0.022087248, 0.0062125106, 0.057943173, -0.033425268, -0.004095124, -0.033183713, -0.013172334, -0.018614886, 0.025091598, 0.025650196, 0.021498457, -0.012274048, 0.00970752, -0.016893802, 0.011624868, 0.0040422836, -0.009730165, -0.010786971, 0.037229773, -0.0025759651, -0.018116677, -0.05984542, -0.021362582, 0.00029699085, 0.015610537, -0.03466324, 0.04764686, 0.030949323, -0.0038328094, -0.00493302, -0.018584691, 0.034270715, 0.006661653, 0.010069853, -0.02023029, -0.07017193, -0.0045518153, 0.018041192, -0.0058426284, -0.058516867, 0.045231305, 0.01958111, 0.06238176, -0.0077297823, 0.030617185, 0.042845942, 0.058426283, 0.01401023, -0.0070579555, -0.06860182, 0.028095948, 0.01613894, 0.037803467, 0.03330449, 0.050454944, 0.014606571, -0.005295354, 0.05211564, -0.043661192, 0.010885104, -0.01613894, 1.3549175e-05, -0.016546564, 0.05293089, 0.032036323, 0.021362582, 0.04613714, -0.0068767886, -0.060539898, -0.03152302, -0.011179499, -0.09402555, 0.010643547, 0.043147884, -0.028111044, 0.03182496, -0.047616668, -0.030979518, 0.03321391, 0.020381263, 0.020411456, 0.031281464, -0.025891751, 0.026857974, 0.013451633, -0.0052802563, -0.007337254, 0.04465761, -0.013013813, 0.013647897, -0.0052349647, -0.0069107576, -0.011783388, -0.02115122, -0.019837761, 0.020018928, -0.040853105, 0.008786588, 0.0048122425, -0.04027941, -0.0016635262, 0.0027854391, -0.040762525, 0.06564276, 0.0061898646, 0.073010206, -0.030722864, 0.01330066, 0.02676739, -0.03750152, 0.026797585, 0.0038346967, 0.031100295, -0.021724915, 0.016395593, 0.003372344, -0.0041970303, 0.0001454288, -0.029213142, 0.020532234, -0.050636113, -0.01265148, -0.025695488, -0.017618468, 0.051813696, -0.022509972, -0.039403774, -0.023808334, 0.005185899, 0.030737963, 0.0239895, 0.019505622, 0.045533247, -0.009556547, -0.009103631, 0.020577526, 0.003963023, -0.018343136, -0.03484441, 0.053715948, -0.019022511, 0.008235539, 0.015504856, -0.028669642, -0.043631, 0.012092882, -0.03025485, 0.009201762, -0.011330471, 0.003876214, 0.02006422, -0.008839428, -0.009148922, 0.04411411, 0.0119570065, -1.7323482e-05, 0.04067194, -0.050968252, -0.010190631, 0.0030703994, 0.031613603, -0.034693435, 0.03526713, -0.00046070144, 0.028065752, 0.042604387, 0.03985669, 0.0104095405, 0.010635999, -0.006238931, 0.0032647762, -0.0010577497, -0.015670925, -0.023793235, -0.031643797, -0.02213254, -0.016455982, 0.032187298, -0.044174496, 0.0218004, -0.032036323, -0.06672976, -0.0041743843, -0.019535817, -0.021060636, -0.04290633, -0.012523153, 0.008461998, -0.00269863, 0.011375763, 0.042755358, -0.0046046553, 0.012621285, 0.012409924, -0.022615653, 0.0006416323, 0.032911964, 0.01248541, -0.03233827, 0.040309608, -0.02568039, -0.0013276128, -0.025755877, -0.0068503683, -0.021136124, 0.010613353, -0.036927827, 0.020456748, -0.02131729, 0.00441594, -0.028926294, 0.0010700163, 0.030843643, 0.0027439217, -0.015323689, 0.010439735, -0.001128518, 0.014674508, 0.008907367, 0.0034440558, -9.3708986e-05, 0.016999481, -0.0190829, 0.0042498703, 0.0051330584, 0.009330088, 0.011315375, -0.041879717, -0.01351957, -0.008771491, 0.049247168, -0.033787604, -0.0006383298, 0.013859258, -0.0027816647, -0.016259717, 0.0057218503, 0.032308076, -0.018388428, 0.021226706, 0.012175917, -0.0057633677, -0.005691656, 0.01521046, -0.016622052, -0.022449583, 0.004083801, 0.0021645655, -0.025287861, -0.023053471, 0.01814687, -0.025770973, -0.026238987, -0.033334687, 0.008930012, 0.021573942, -0.00937538, 0.005004732, -0.0042385478, -0.0081751505, -0.036716465, -0.05903017, 0.025257668, -0.024834944, 0.0054878434, 0.024246152, -0.014198946, 0.026510738, -0.020260483, -0.0037629849, 0.01177584, 0.0030590764, 0.033666823, -0.021619234, -0.014168751, 0.0071409903, 0.015059488, -0.034814216, 0.013104397, 0.037410937, 0.004514072, -0.012817549, 0.00733348, -0.006405, -0.010620902, 0.003974346, -0.016833412, 0.015104779, -0.00066050387, 0.0018031755, -0.032730795, -0.0014512213, -0.028397892, -0.005521812, 0.0032704377, -0.016546564, -0.00796379, -0.0047027874, -0.03831677, 0.0899795, -0.0034138614, 0.0075637125, 0.009783005, 0.00030737018, 0.0024570744, -0.016184231, 0.065521985, -0.052659143, 0.026299376, -0.017754344, -0.029394308, -0.006525778, 0.013836612, 0.024382029, 0.013036459, 0.023415806, -0.014146105, -0.03282138, 0.021136124, 0.015089682, 0.025438834, 0.0037327902, -0.019097997, 0.026812682, -0.07047387, -0.007533518, 0.025499223, -0.027280696, 0.014432953, -0.057550643, 0.008424255, 0.017286329, -0.007023987, -0.016923996, -0.000671355, 0.008499741, -0.0179959, 0.008182699, -0.016923996, -0.015776606, 0.016199328, -0.043902747, 0.043963138, 0.0028250692, 0.038195994, 0.0053217737, -0.07005115, 0.0020079317, 0.0010690726, 0.032851573, -0.019490525, -0.014168751, 0.0043970686, 0.00078269705, 0.041517384, 0.0049103745, -0.040158633, -0.048703667, 0.023898916, -0.011088916, 0.024563195, -0.018509205, 0.013481827, -0.007835463, 0.041789137, 0.025937043, -0.006491809, -0.02992271, 0.034059353, -0.028971586, 0.013232723, 0.043540414, -0.025967237, -0.00899795, 0.009964173, 0.014380112, 0.015006647, 0.014523536, 0.008348769, 0.011768292, -0.035569075, -0.007812817, 0.025438834, -0.00087752653, 0.01924897, -0.014644314, 0.013836612, -0.029847225, 0.014478245, -0.011262534, 0.011896618, 0.02764303, 0.01881115, -0.02768832, 0.02545393, -0.031975936, -0.033123326, -0.00087233685, 0.01592758, 0.0104095405, -0.013964939, -0.008250637, -6.899906e-05, 0.0392528, 0.014704702, 0.011202145, 0.017950607, 0.016154038, 0.007835463, -0.03445188, -0.0052462877, -0.016954191, -0.0020400134, 0.010802069, 0.03336488, -0.010477479, -0.0039328285, 0.0075788097, 0.00016087986, 0.025393542, 0.020622818, 0.04725433, -0.053836726, 0.02087947, -0.0071183443, 0.016818315, 0.009896235, 0.0022268416, 0.009941527, -0.011330471, -0.030073684, -0.0033459237, 0.01128518, -0.020532234, -0.02883571, -0.032730795, -0.012357083, -0.028760225, -0.012560896, -0.022962889, -0.032398656, 0.005223642, -0.015134974, 0.00268542, -0.10966629, -0.007692039, 0.033002548, -0.028246919, -0.018418621, -0.01958111, 0.036384325, 0.010115145, 0.0034648145, 0.018056288, 0.01057561, -0.025438834, 0.0010256681, 0.0031213525, 0.038407356, -0.061687287, 0.0048235655, -0.0090130465, 0.04655986, -0.016788121, -0.011579576, 0.02246468, 0.010439735, -0.0043480024, -0.019067803, -0.02480475, -0.020305775, -0.02436693, 0.024819847, -0.0077599767, -0.016395593, 0.018675275, 0.008741297, 0.0103038605, -0.006578618, 0.02131729, -0.01270432, -0.013889452, 0.018931927, -0.010983235, 0.014440501, -0.01025102, 0.019369747, -0.011353117, 0.039192412, 0.005895469, 0.017074969, 0.0005727512, -0.011700354, 0.015640732, -0.014810383, 0.024336737, -0.017754344, 0.014682057, -0.004487652, -0.027794002, -0.0020456747, -0.039554745, -0.0037554363, 0.015195363, 0.0011700355, 0.011911715, -0.006412549, -0.026072918, 0.0059785033, -0.0013946067, 0.009307443, 0.022011762, -0.023430903, -0.006635233, -0.037441134, 0.024880236, -0.017799634, -0.022721332, 0.013172334, 0.017844927, 0.02638996, 0.033515852, 0.0031213525, 0.016471079, 0.05806395, -0.010500124, -0.022162735, 0.012508056, 0.00937538, 0.027552446, 0.010386894, -0.013647897, 0.012772257, 0.05718831, 0.03411974, 0.003104368, -0.010092499, -0.012870389, -0.03931319, 0.011141757, 0.019113094, 0.032368463, 0.026103113, -0.015716217, 0.022011762, 0.015897384, -0.019520719, -0.012591091, -0.040520966, 9.718842e-05, 0.0018663951, -0.014833029, -0.0088847205, -0.0023362965, 0.010749228, 0.04040019, 0.017603371, 0.019188581, -0.009986818, 0.024850043, 0.04372158, -0.006854143, -0.008590325, 0.0033157293, 0.009503707, 0.011171951, -0.01793551, 0.019128192, -0.022751527, 0.018931927, 0.0025854008, 0.020215193, 0.008922463, 0.026193695, -0.016108746, 0.018886635, -0.013806418, 0.0049820864, -0.00043664023, -0.0061974134, -0.025831362, -0.02196647, -0.004423489, -0.0011634304, -0.019686788, 0.003642207, 0.00056614616, -0.03547849, -0.031009711, 2.6066313e-05, -0.014032876, -0.04840172, 0.014493342, 0.021000247, 0.035448298, 0.0122363055, -0.013089299, 0.025891751, 0.006144573, 0.010515221, 0.024019694, -0.035750244, -0.040551163, 0.026903266, -0.0010756777, 0.049518917, 0.01958111, -0.025333153, 0.08031727, -0.02643525, -0.02562, -0.019807568, -0.003068512, 0.022645846, -0.02693346, 0.010417089, 0.03185516, 0.018796053, 0.04806958, 0.028262015, -0.020924762, 0.038226187, 0.0063974517, 0.03396877, 0.022645846, 0.010069853, -0.0010785084, 0.0069673723, 0.0022740203, -0.033062935, -0.02611821, 0.016773023, -0.032398656, 0.0022343902, 0.026254084, 0.020184997, 2.1156764e-05, 0.05338381, -0.0010596369, -0.012387278, -0.00010137555, 0.0022910049, -0.022887401, 0.004257419, -0.0376223, 0.03297235, 0.008643164, -0.043147884, -0.021332387, 0.059301924, -0.00047226026, -0.0042498703, -0.010930395, -0.020683207, -0.032911964, 0.01973208, 0.00246651, 0.022117443, -0.025937043, 0.020622818, -0.016773023, -0.022751527, -0.03593141, -0.01231934, -0.034089547, -0.028699836, -0.0127949035, 0.0039026341, 0.024321638, 0.017512787, 0.0033515852, -0.030828545, -6.162737e-05, -0.0084544495, -0.010719034, -0.031553213, -0.010447284, 0.0029835904, -0.006046441, 0.029620767, 0.008280831, -0.007752428, 0.026012529, 0.0013861145, 0.06123437, 0.0033100678, 0.002294779, -0.01576151, 0.011194597, -0.022781722, 0.00711457, 0.024759458, 0.017150454, -0.008756394, 0.00034629274, 0.040853105, 0.005261385, -0.018131774, 0.03445188, 0.015535051, 0.037169382, 0.045352083, 0.02196647, 0.024744362, -0.0044348114, 0.04858289, -0.010137791, -0.020819081, 0.0068013025, 0.0007558051, -0.038226187, -0.024955723, 0.018690372, 0.0066767503, -0.022072151, 0.026631515, -0.02327993, 0.016033258, -0.006155896, -0.0064200973, 0.015165169, -0.015255752, 0.010719034, 0.0020758694, 0.0026533382, 0.013708286, -0.009262151, -0.013753577, 0.0073599, -0.007344803, -0.022585457, -0.029379211, -0.0017097614, -0.0008110044, 0.01160977, -0.0067031705, 0.0056727845, -0.009352734, -0.0045706867, 0.00036657962, -0.007612779, -0.010205728, -0.0027061787, -0.016018162, 0.01243257, -0.009413123, -0.011277632, -0.049035806, -0.011858875, -0.0047971453, 0.033274297, -0.024925528, -0.014599022, 0.026012529, -0.026208794, 0.0060049235, -0.006854143, 0.011473895, -0.0028741353, 0.009428221, 0.0035214291, -0.018660177, -0.008930012, 0.0024853817, 0.004623527, 0.046378694, 0.016561663, -0.005752045, -0.0143423695, -0.028639447, -0.016773023, 0.011760742, 0.00067607284, -0.012372181, -0.0013974374, -0.0035214291, 0.024880236, 0.022374095, -0.052477974, -0.00050670083, -0.046861805, -0.017603371, 0.033062935, -0.02294779, -0.005880371, 0.013655446, -0.01052277, -0.009126276, 0.009858492, 0.0042951624, 0.0028533766, 0.0020626592, -0.011647514, 0.01684851, 0.07790171, -0.057309087, -0.030300142, 0.00027882698, -0.027794002, 0.011262534, -0.020411456, 0.025227472, -0.029575475, 0.005197222, -0.005706753, 0.030013295, -0.017241038, 0.032308076, 0.034089547, 0.015580342, -0.013866806, 0.0013285563, 0.022993082, 0.003838471, 0.0054161316, 0.015670925, -0.011926812, -0.017890219, 0.02480475, 0.035236936, 0.055467226, -0.016727732, 0.024925528, 0.03478402, -0.015776606, -0.032760993, 0.02115122, 0.006756011, -0.033334687, 0.019067803, -0.018086482, 0.0017314637, -0.027492058, -0.0010181195, -0.0196264, 0.004321582, -0.015746411, 0.00011712149, 0.0070466325, 0.016305009, -0.006231382, 0.01155693, 0.008107213, -0.002674097, -0.009971721, -0.017467497, -0.03505577, -0.008628068, 0.0013162898, -0.031070102, 0.006722042, 0.017965704, 0.009126276, 0.016561663, -0.011889069, 0.03892066, -0.030179365, 0.008484644, 0.0343613, -0.034723632, -0.022962889, 0.016229523, -0.004076252, 0.026269183, -0.032791186, 0.0023136507, 0.005042475, -0.0023344094, 0.040309608, 0.007518421, -0.035901215, -0.009873589, 0.019264067, 0.018494109, -0.0030949323, -0.018448817, -0.009609387, 0.0062955455, -0.013421439, 0.0036554171, 0.027114626, -0.0021683397, -0.027054237, 0.023370514, -0.004072478, 0.012900583, 0.00692208, 0.025091598, 0.011368215, 0.0038195995, -0.00083506556, -0.013768675, 0.0059785033, -0.0053519686, 0.0013898888, -0.0147198, -0.0048386627, -0.032700602, -0.0084770955, 0.006891886, -0.028881002, -0.024321638, 0.003981895, -0.0013766787, 0.019294262, 0.0049066, -0.02115122, 0.0078052683, -0.0060200207, 0.0037592105, 0.020562429, 0.0719836, -0.021740012, -0.038346965, 0.021634331, -0.013693188, -0.01847901, -0.017588275, -0.0003854512, -0.00042390195, 0.027582642, 0.0010152888, -0.030632282, -0.00779772, -0.031794768, 0.01504439, 0.0120853335, -0.027778905, 0.006589941, 0.008318574, 0.0072051533, 0.00134271, -0.07150049, 0.0020702078, 0.013708286, 0.03864891, 0.0030232205, -0.0045216205, -0.026752293, -0.034753826, 0.026148403, 0.0038686653, -0.021619234, -0.025710585, -0.0075863586, -0.0071711848, 0.017090065, -0.006940952, -0.003678063, -0.039373577, 0.016591856, -0.013662994, -0.0019296148, -0.019822665, -0.00417061, 0.014931161, -0.03384799, 0.0025948365, 0.0041517387, 0.02311386, -0.012915681, 0.029801933, 0.03547849, 0.019777372, 0.0035006704, 0.0047254334, -0.00013988528, 0.021468263, 0.020577526, 0.0003026523, 0.0076731676, 0.00366674, -0.012900583, 0.029968003, -0.009420672, 0.010786971, -0.017361816, -0.035961606, -0.008484644, -0.0073900945, 0.0019758502, -0.006744688, -0.024910431, -0.027416572, -0.015142523, -0.05773181, -0.0023721524, 0.010568062, 0.018433718, 0.040098246, -0.004068704, 0.023944208, 0.0040536067, -0.008258185, 0.0001574594, -0.024321638, -0.014742446, -0.027280696, -0.050847474, 0.0288961, 0.00023565834, -0.0063106427, 0.0069485004, 0.008311026, -0.018222358, 0.0045216205, -0.042694967, 0.022343902, -0.014493342, 0.0061068297, 0.008348769, -0.00899795, -0.0020626592, -0.01674283, -0.015806802, -0.010862458, 0.025499223, 0.0021853242, 0.0019399942, 0.012726965, -0.003211936, -0.026525835, 0.0381658, -0.0006425759, 0.0074504833, -0.0067786565, 0.011405958, 0.03069267, -0.007356126, -0.018705469, -0.03297235, 0.04215147, 0.031643797, 0.0062728995, -0.026344668, 0.03735055, -0.009941527, 0.009533902, -0.008952658, 0.0071447645, 0.013678092, -0.009956623, -0.032610018, 0.02321954, -0.004782048, -0.00048028067, -0.037078798, 0.007522195, -0.03254963, -0.015610537, -0.029817032, 0.003102481, 0.0019286713, 8.49809e-05, -0.044355664, -0.02066811, 0.003238356, 0.022600554, 0.012877938, 0.03985669, 0.027718516, -0.03511616, -0.033274297, -0.014651863, -0.03042092, 0.012568445, -0.00013858786, 0.020970054, 0.031311657, 0.01755808, 0.011345569, -0.0020796435, -0.005812434, -0.018297844, 0.010500124, -0.020018928, -0.019097997, 0.00011429076, 0.01897722, 0.029756643, -0.00055482326, -0.002649564, 0.0070353094, -0.012598639, 0.0030515278, 0.03550869, 0.012455216, -0.0101302415, -0.0151576195, 0.005702979, -0.0021740012, 0.0075486153, -0.028714933, 0.024593389, -0.039222606, -0.015625633, 0.026510738, -0.0024533, 0.020351067, 0.012825098, 0.013791321, 0.03233827, -0.008741297, 0.0016993821, 0.024004597, 0.0019966087, -0.008960207, 0.014138557, 0.019128192, -0.00877904, -0.007752428, -0.0026816456, -0.017890219, 0.022328805, 0.034482077, 0.025393542, 0.015361432, -0.0062728995, 0.027522251, 0.021936277, -0.027024044, 0.0116777085, 0.02420086, -0.013059105, 0.016471079, -0.021785304, 0.05320264, -0.00438952, -0.0026231438, -0.023189347, 0.0035742694, 0.030028392, 0.022555264, 0.005702979, -0.007005115, -0.024382029, -0.0078052683, -0.027054237, -0.020622818, 0.05175331, 0.004076252, -0.041215442, 0.000604361, 0.0056954306, 0.02900178, 0.037924245, -0.010703937, 0.00806947, 0.00970752, 0.018418621, 0.005276482, 0.021875888, 0.009511256, -0.031885352, -0.0403398, -0.0046197525, -0.011368215, -0.022509972, -0.00064965273, 0.059966203, -0.026450349, -0.03729016, -0.03085874, -0.026133306, 0.0047933706, 0.031492822, 0.013444084, 0.0073938686, -0.017724149, 0.003691273, -0.018252552, -0.022751527, 0.017980803, -0.015014196, 0.052659143, -0.04393294, 0.01810158, -0.017844927, 0.015391627, -0.0038950855, 0.002232503, 0.0056123957, -0.006148347], "content_hash": "66167bb880c23493d664ad623f6f1c5a"}, {"content": "is expected behavior for small models. The dashboard is designed for exploring *how* the model works, not for getting useful outputs. Try different prompts or a different model. ### BertViz visualization is hard to read **Why**: With 12+ heads selected simultaneously, the attention lines overlap and become a dense mess.\n\n**Fix**: Double-click on a single head in the BertViz visualization to isolate it. Then explore heads one at a time. Use the head categories to guide which heads to investigate.\n\n### The dashboard becomes slow or unresponsive\n\n**Why**: Larger models require more memory and computation. Running multiple experiments without refreshing can also accumulate memory usage.\n\n**Fix**: Try a smaller model. Refresh the browser page if things get sluggish. Close other memory-intensive applications.\n\n## Frequently Asked Questions\n\n### Which model should I start with?\n\n**GPT-2 (124M)** is the best starting model. It's small, fast, well-studied, and has clean attention patterns that are easy to understand. Move to Qwen2.5-0.5B once you're comfortable for a comparison.\n\n### What prompts work best for learning?\n\nStart with short, simple prompts (5-10 words) that have clear, predictable continuations:\n- \"The cat sat on the\" (predict a location)\n- \"The capital of France is\" (predict a fact)\n- \"Once upon a time there was a\" (predict a story element)\n\nThese give clear, interpretable results in the pipeline and experiments.\n\n### Can I use my own model?\n\nYes! Type any HuggingFace model ID into the model dropdown. The dashboard supports GPT-2, LLaMA, OPT, GPT-NeoX, BLOOM, Falcon, and MPT architectures. Unknown architectures may need manual configuration in the sidebar.\n\n### What's the difference between the pipeline and the investigation panel?", "source_file": "troubleshooting_and_faq.md", "chunk_index": 1, "embedding": [-5.0072726e-05, -0.012094242, 0.055194624, -0.00953762, -0.037355088, 0.0028744242, -0.035338197, 0.025466794, 0.008827447, 0.060336277, -0.028406909, -0.034571208, -0.015467562, -0.048234932, -0.018180422, -0.0024607484, 0.0022334931, 0.008145682, -0.0320714, 0.055847984, -0.0103969285, -0.012797313, -0.003634309, 0.032980423, 0.021986948, 0.022057965, -0.005368906, 0.014395202, 0.024429942, 0.008621497, 0.030395392, -0.034116697, -0.06312015, -0.030026102, -0.03684376, 0.015084069, -0.014572744, 0.02968522, -0.009835892, 0.067892514, -0.030281765, -0.017740116, -0.029372744, 0.05036545, -0.019501343, 0.040280998, 0.013890979, -0.024870249, -0.05772284, 0.037440307, -0.0016555901, 0.036786947, -0.04905873, 0.005830518, 0.0104679465, -0.027895585, -0.03650288, -0.027923992, -0.01102188, 0.0022086373, 0.019714395, 0.00419357, 0.0037852207, 0.02968522, 0.014167946, -0.025083302, -0.037525527, 0.0077266796, -0.013841267, -0.03724146, -0.020737045, -0.0021731285, 0.0100489445, -0.01200192, 0.0012268234, -0.0026223129, 0.0024394433, 0.03638925, -0.0066862763, -0.045905564, -0.027980806, -0.020623416, 0.018606525, -0.03880384, -0.012364107, -0.014743186, -0.027085988, -0.019004222, -0.021546641, -0.05655816, -0.0006267274, 0.009367178, -0.022356238, 0.025836084, 0.034968905, -0.009381382, 0.025637235, -0.053745873, -0.0028176103, 0.04496814, 0.03704261, -0.0213762, -0.0061500957, 0.0032224087, 0.03408829, -0.03300883, 0.022327831, 0.018492898, -0.028435316, -0.0062388675, -0.079255275, -0.021021113, -0.0109082535, 0.0197286, -0.0129748555, 0.0042219767, -0.048859883, -0.010752015, 0.02762572, -0.064938195, 0.008834549, 0.017697504, 0.0074426103, 0.009161228, 0.0057808063, -0.0017408109, 0.011313052, -0.028364299, 0.016802687, -0.001802951, -0.008294818, 0.037923224, 0.025196929, -0.013890979, -3.7422775e-05, -0.015694818, -0.033576965, -0.0050741844, -0.03346334, -0.0023045104, 0.00928906, -0.017498657, 0.0066294624, 0.019600768, -0.067722075, 0.015978886, -0.012136852, 0.02602073, -0.060790785, 0.038945872, 0.029145489, 0.030991938, -0.019558158, -0.051018808, -0.0105815735, -0.010943762, 0.014025912, 0.035252973, -0.019799616, -0.068915166, 0.032383878, 0.018634932, -0.020950096, -0.034855276, -0.0427524, 0.07738042, -0.034343954, 0.0072295587, -0.023861803, -0.033804223, 0.013315739, 0.01874856, 0.042042226, -0.049967755, 0.03363378, 0.0041509597, -0.057467178, -0.047865644, 0.006015163, -0.04218426, -0.038917467, -0.0030714972, -0.012122649, -0.011810172, -0.04505336, -0.007471017, -0.041104797, 0.021063723, 0.0055322456, -0.005645873, -0.001422121, 0.030310173, 0.05965451, 0.023322072, -0.0061110365, -0.026773512, 0.06721075, 0.018535508, 0.01842188, 0.015879462, 0.0024927063, 0.009935317, 0.05891593, -0.0814142, 0.011341458, 0.023165835, -0.0002512236, 0.01056737, -0.041502494, -0.031219194, 0.0055713053, -0.005368906, 0.027100192, 0.019146256, -0.06118848, -0.008997888, -0.052268714, 0.0017949616, 0.029884068, 0.04954165, 0.020623416, 0.036900576, 0.041502494, -0.0024714011, -0.025907101, 0.035906333, 0.040650286, -0.037724376, -0.028520538, 0.013088483, 0.027284836, -0.011007678, -0.024387332, 0.026205374, 0.019998465, -0.023705566, 0.01056737, -0.018379271, -0.0043036467, -0.050905183, 0.031815737, 0.01994165, -0.025154319, -0.036076777, 0.08010749, 0.0059405947, -0.03548023, -0.076357774, 0.024685604, 0.01718618, 0.020864874, -0.0062459693, -0.034968905, -0.0052268715, -0.0023329174, -0.016603839, -0.03573589, 0.0044953935, -0.039684452, -0.01820883, -0.009502111, 0.022299424, -0.035792705, 0.02310902, -0.019643378, -0.027157005, -0.0493428, -0.04337735, -0.0764714, -0.036985796, 0.01887639, -0.011590019, -0.0043604607, 0.022455662, -0.03795163, -0.00965835, 0.02273973, 0.007584645, -0.027725143, -0.021575049, -0.02113474, -0.022171592, 0.033946257, 0.013940691, -0.009502111, 0.017129367, 0.008394241, -0.01870595, 0.0069135316, 0.006675624, 0.00916833, -0.010659693, 0.008742226, -0.017413436, 0.0053547025, 0.020041075, 0.011014779, 0.0143241845, 0.0013946017, 0.0029365642, 0.012847025, -0.015836852, -0.005759501, -0.0037497121, 0.040934358, 0.020779654, -0.00312476, 0.016419193, -0.0013892754, 0.008848752, -0.038207293, -0.023378886, -0.010297504, 0.019515546, 0.006902879, -0.016390787, -0.00785451, 0.035395008, 0.033037234, -0.017626487, -0.011696545, -0.048206527, -0.0349405, 0.026787715, 0.009502111, 0.015694818, 0.0043036467, 0.014033013, -0.026418425, -0.02458618, 0.021717083, 0.0062140115, -0.009203839, 0.04008215, -0.011618426, -0.051075622, -0.003515355, 0.019970058, -0.0074639153, 0.06874472, -0.083345875, -0.004424376, -0.029514778, 0.0024323417, -0.04181497, 0.043718234, 0.0024696258, 0.027852975, 0.04329213, -0.017058348, -0.008337428, 0.005106142, -0.027355853, -0.028378503, 0.03775278, 0.05340499, -0.025154319, -0.016064107, -0.029287525, -0.011085796, -0.020566603, -0.0035260075, 0.014700576, 0.013564299, -0.0058979844, 0.011519002, -0.0146721685, -0.011242035, 0.00937428, -0.006128791, 0.015694818, 0.017058348, 0.0077976966, -0.015467562, -0.04925758, -0.02688714, -0.025665643, 0.028421113, -0.012172361, 0.05772284, 0.033150863, 0.010254894, 0.012641075, -0.016191939, 0.0075704413, 0.0148, 0.033292897, -0.014913628, -0.052183494, 0.0044705374, 0.018862188, -0.014452015, -0.011043186, 0.0044385795, 0.012946449, 0.079084836, -0.0037923225, 0.0097932825, 0.035792705, 0.04155931, 0.015907869, -0.015595393, -0.060506716, -0.0022832053, 0.0050812857, 0.028889827, 0.037610747, 0.029074471, 0.009850096, -0.041928597, 0.03712783, -0.05331977, 0.013436468, -0.002150048, -0.0296, -0.0041509597, 0.03724146, 0.019330902, 0.033861037, 0.05542188, -0.0064803264, -0.06800614, -0.0522119, -0.004658733, -0.08760691, 0.02150403, 0.04263877, -0.025466794, 0.0076769674, -0.033974662, -0.009693858, 0.02931593, 0.057523992, 0.0025140115, 0.02256929, -0.06527908, 0.03704261, 0.013173704, 0.020779654, -0.00042632557, 0.041388866, -0.019884836, 0.035224568, 0.017725911, -0.022299424, -0.031219194, -0.046587333, -0.038349327, -0.008309021, -0.05760921, -0.0010146593, 0.012747601, -0.0073644915, 0.028350096, -0.0018171545, -0.043434165, 0.08152783, -0.017228791, 0.07993704, 0.006593954, 0.014558541, 0.058234163, -0.010503455, 0.041019578, -0.012726296, 0.04374664, 0.0014407629, -0.0004365343, -0.013841267, -0.0374119, 0.0029259117, -0.030821497, 0.029259117, -0.031190787, -0.05377428, -0.048945107, -0.017228791, 0.02400384, -0.016248751, -0.048206527, 0.0038988483, 0.028051823, 0.04181497, 0.045536276, 0.03732668, 0.07328983, -0.021035317, -0.029344337, 0.012399616, 0.023932822, -0.03851977, -0.020495586, 0.022682916, -0.016561229, 0.027654126, -0.011817275, -0.016944721, -0.04144568, -0.022256814, -0.0037461612, -0.0038455853, 0.0069206334, -0.012520345, 0.052808445, 0.0031300862, 0.0008965931, 0.0050422265, 0.011696545, -0.013422265, 0.038747024, -0.03474165, -0.028364299, -0.0097932825, 0.03491209, -0.014601151, 0.03195777, 0.015652208, 0.04707025, 0.029912475, 0.022824952, -0.0068638194, 0.016476007, -0.020765452, -0.01245643, 0.0053866603, -0.018265642, -0.030452207, -0.055393472, -0.0151124755, -0.008784837, 0.007918426, -0.044456813, 0.023009596, -0.026290596, -0.060961228, -0.0148, -0.019870633, -0.010766218, -0.03573589, 0.0010022313, 0.023719769, 0.02104952, 0.006377351, 0.029713627, -0.021518234, 0.022327831, 0.019799616, -0.029997697, -0.0027945298, 0.034258734, 0.02717121, -0.030679462, 0.027810365, -0.011298848, 0.025381574, 0.0057985606, -0.008983685, -0.016348176, 0.018862188, -0.024245298, 0.0065761996, 0.016518617, 0.019416122, -0.021276776, 0.0020364204, 0.001590787, 0.0146721685, -0.044002302, -0.008024952, 0.0004953455, -0.016902111, -0.012477735, 0.00066134834, -0.015652208, 0.023733974, -0.01998426, -0.025935508, 0.03269635, -0.01714357, 0.00081936177, -0.031389635, -0.02310902, -0.017910557, 0.02289597, -0.018038388, 0.025665643, -0.0024767274, -0.0033608924, 0.0018695297, 0.00021194217, 0.028321689, -0.03548023, 0.0118598845, 0.010340115, -0.0304238, -0.01887639, 0.028960845, -0.03235547, -0.052638, -0.02565144, -0.0026453934, -0.017200384, -0.009381382, 0.0009072457, -0.029287525, -0.011710748, -0.03079309, 0.009154127, 0.006689827, -0.03428714, -0.0046019196, -0.014544338, -0.017086755, -0.029940883, -0.06556315, 0.015893666, -0.03502572, -0.00063738006, 0.022398848, -0.010666794, -0.001171785, -0.01862073, 0.011909597, -0.0052410746, -0.018350864, 0.008784837, -0.0028176103, -0.015936276, 0.008841651, 0.028222265, -0.024415739, 0.0002412368, 0.048320152, -0.0036609406, -0.0071265833, 0.01994165, 0.009395585, -0.0077550863, 0.0009702735, -0.005173608, 0.012641075, -0.013202111, -0.016390787, -0.04229789, 0.013734741, -0.01994165, 0.008429751, 0.02146142, -0.021262571, -0.0033786467, -0.04420115, -0.027412668, 0.06192706, 0.012009021, 0.014984645, 0.0068176584, 0.007932629, -0.015268714, -0.028463723, 0.057552397, -0.039371975, 0.03851977, -0.01846449, -0.040337812, 0.0055642035, 0.021972744, 0.0035260075, 0.018009981, 0.044684067, -0.012506142, -0.027611516, -0.013166603, 0.033292897, 0.025750864, -0.011866987, -0.029117081, 0.015240307, -0.041616123, -0.0066259117, 0.034713242, -0.027867178, 0.029628407, -0.052183494, 0.017314011, 0.028179655, 0.010510556, -0.020765452, 0.00022725528, 0.021248369, 0.015069866, 0.008294818, -0.026319, -0.014452015, 0.0026951055, -0.045905564, -0.00022015355, 0.0012676583, 0.04715547, 0.0047581573, -0.06652898, -0.008600192, -0.011206525, 0.015467562, -0.015353935, 0.002709309, -0.014629559, -0.010283302, 0.03195777, 0.004498944, -0.021688676, -0.031503264, 0.016717466, -0.007240211, -0.022214204, -0.02569405, -0.010404031, -0.01553858, 0.027739348, 0.007968138, -0.006924184, -0.044456813, 0.03309405, -0.030537428, 0.017853742, 0.034713242, -0.022725528, 0.0005206454, -0.0009578455, -0.020282533, 0.02890403, 0.009523416, 0.030537428, 0.011043186, -0.043064874, 0.0103046065, -0.0012143954, 0.0004922385, 0.017938964, -0.01558119, 0.000193744, -0.020893281, 0.006821209, -0.024756622, 0.021361995, 0.02651785, -0.01155451, -0.028932437, 0.0365881, -0.022881765, -0.04596238, 0.0021873321, 0.013926487, -0.02204376, -0.010609981, -0.003108781, -0.02504069, 0.04226948, -0.012009021, 0.020651823, 0.032383878, 0.024486756, 0.016262956, -0.028776199, 0.015552783, -0.02310902, -0.0014274472, 0.034457583, 0.026588866, 0.010666794, -0.016476007, 0.0042610364, -0.038292512, 0.012733397, 0.008507869, 0.05496737, -0.057495587, 0.033520155, 0.0110005755, 0.004062188, 0.018379271, -0.010190979, 0.0119238, 0.030310173, -0.03567908, 0.009296161, -0.006437716, -0.01935931, -0.028833013, -0.032525912, -0.012761804, -0.060961228, -0.0062459693, -0.005312092, -0.0138341645, 0.0052481764, -0.029514778, 0.007272169, -0.12658119, -0.012080038, -0.0009143474, -0.02113474, -0.00032357246, -0.01710096, 0.042468328, -0.0082593085, 0.0040550865, -0.004218426, 0.0133015355, -0.0213762, 0.025069097, 0.008230902, 0.019813819, -0.07158541, 0.017299809, -0.008621497, 0.051246066, -0.01994165, -0.027483685, 0.023947025, 0.00875643, 0.021532437, -0.038832244, -0.023322072, -0.04155931, 0.017385028, 0.049740497, -0.0076414584, -0.03519616, 0.04871785, 0.0076982724, 0.0092819575, 0.004917946, 0.03420192, -0.00019008217, -0.015055662, 0.014728983, -0.026304798, -0.0374119, -0.025807677, 0.014466219, -0.03712783, 0.036758542, 0.0042077736, -0.0061856047, 0.0064625717, -0.042525142, 0.018720154, 0.0035988004, 0.024302112, -0.021617658, 0.052552782, 0.020893281, -0.024685604, 0.017796928, -0.016419193, 0.012790211, 0.010617082, 0.0016085412, 0.033037234, 0.0049037426, -0.017299809, -0.004037332, 0.0025140115, 0.031815737, 0.006668522, -0.036730133, -0.017385028, -0.05073474, 0.035451822, -0.0050493283, -0.0009933541, 0.000117178504, 0.0055073895, 0.02293858, 0.030480614, 0.0028264874, 0.0044208253, 0.020452974, -0.009473705, -0.042695586, 0.018350864, 0.009693858, 0.042042226, 0.017754318, -0.025665643, 0.03326449, 0.024657197, 0.010482149, 0.005649424, -0.012080038, -0.009509213, -0.024202688, 0.00994952, -0.019600768, 0.026546257, 0.030480614, -0.0069525912, 0.033690594, 0.009132821, -0.024614587, -0.012357006, -0.011661036, 0.0010235364, 0.0064554703, -0.009409789, -0.016831094, -0.0034851728, 0.020140499, 0.03786641, 0.009388483, 0.036815356, -0.021575049, 0.025211133, 0.052723225, 0.015368138, 0.022924377, -0.011149712, 0.023052206, -0.02315163, -0.020296738, 0.016135124, -0.016532822, -0.0048327255, 0.013038771, 0.029429559, -0.012271785, 0.018223032, -0.0047190976, 0.029259117, -0.021603454, 0.007598848, -0.0028602206, -0.0017789827, -0.004665835, -0.01516929, 0.0039166026, -0.0014203455, -0.03107716, 0.022498272, 0.010411132, -0.01681689, -0.01282572, 0.004033781, -0.018492898, -0.051984645, 0.023421496, 0.018975815, 0.042070635, 0.00037261876, -0.012392514, 0.020552399, 0.0038988483, 0.0010182102, 0.0017177303, -0.053632244, -0.041218426, -0.002734165, 0.027938195, 0.03326449, 0.016532822, -0.026915547, 0.03133282, -0.03684376, -0.03141804, 0.018265642, -0.00661881, 0.035338197, -0.028676774, -0.0003890415, 0.023691362, 0.0017585652, 0.03999693, 0.046274856, -0.00485048, 0.011270441, 0.011803071, 0.011313052, -0.017541267, 0.03641766, 0.01155451, 0.0051167947, 0.015481765, -0.013422265, -0.024188483, 0.0069206334, -0.014494626, -0.009516315, 0.015893666, 0.0013670825, -0.00674309, 0.042468328, 0.014380998, 0.0031052304, 0.00040080375, -0.01229309, -0.021717083, 0.020538196, -0.01952975, 0.03215662, -0.009012092, -0.017938964, -0.01957236, 0.04190019, 0.0027199616, -0.01846449, -0.021518234, -0.015510173, -0.035451822, 0.017569674, -0.010077351, 0.022313626, -0.0028424663, 0.013010365, 0.02881881, 0.0011043187, -0.034968905, 0.004353359, -0.029401151, -0.034599617, -0.022128982, 0.014295777, -0.016532822, 0.019757006, 0.012172361, -0.0010892274, -0.0047013434, 0.013585605, -0.008159885, -0.02330787, 0.009906909, -0.0101412665, -0.016149327, 0.01163263, 0.012151056, -0.014515931, 0.017228791, 0.014167946, 0.045848753, 0.0156238, -0.005447025, -0.02315163, 0.000531298, -0.012776008, 0.012101344, -0.009651247, 0.01146929, -0.027781958, -0.009303262, 0.03621881, 0.017129367, -0.02273973, 0.024358924, -0.017271401, 0.026958156, 0.029742034, 0.017015738, -0.011320153, -0.008948176, 0.048178118, -0.012108445, -0.0226119, -0.0038739922, 0.0028637715, -0.009871401, 0.0013537668, 0.03474165, 0.004410173, -0.017001536, 0.028364299, -0.028591555, 0.010020537, -0.011703647, 0.018350864, 0.0062885797, -0.0068070055, 0.011497697, -0.00928906, 0.01413954, -0.0036609406, 0.009097313, -0.013372553, -0.003884645, -0.021163147, -0.040877543, -0.012683685, 0.010091554, -0.020396162, 0.0070449137, 0.017214587, -0.012562956, -0.010617082, 0.005209117, 0.003165595, -0.011795969, -0.01994165, 0.021702878, -0.05093359, 0.022654511, 0.0023453454, -0.014508829, -0.054370824, 0.013024568, -0.013422265, 0.033037234, -0.029628407, 0.010723609, 0.03005451, -0.028350096, 0.010574472, 0.00661881, 0.023563532, 0.009360077, 0.021930134, 0.016859502, -0.015311324, -0.016788483, -0.011277543, -0.010425336, 0.022185797, -0.0017541266, -0.013017466, -0.015226103, -0.023620345, 0.002293858, 0.020424567, 0.009928215, -0.013060076, 0.008046257, 0.0011087572, -0.0006484765, 0.019757006, -0.054200385, -0.028918235, -0.060733974, -0.020765452, 0.033236083, -0.008380039, 0.0134648755, 0.020168906, 0.00076565496, -0.01052476, 0.026702495, 0.0016112044, 0.018819578, 0.0044208253, 0.014871017, 0.021915931, 0.056387715, -0.029074471, -0.029372744, 0.0002787428, 0.004836276, 0.029628407, 0.001809165, 0.026546257, -0.006668522, -0.00604357, 0.0014167946, 0.030196546, -0.01389808, 0.030565834, 0.036161996, -0.006459021, -0.0041119, -0.026943954, 0.029202303, 0.0012099568, 0.016802687, 0.011206525, -0.0094452975, 0.0006924184, 0.023052206, 0.038576584, 0.030849904, -0.021717083, 0.017356621, 0.048746258, -0.009381382, -0.036360845, 0.021077927, 0.014743186, -0.022597697, 0.006899328, -0.01171785, 0.031730518, -0.029117081, 0.008252207, -0.0028477926, -0.029543186, 0.01105739, -0.022327831, 0.017072553, 0.014047217, 0.020367755, 0.02495547, 0.016092515, 0.0006018714, -0.003959213, -0.009203839, -0.01796737, -0.021958541, 0.01377025, -0.017115163, -0.007350288, 0.033946257, 0.026787715, 0.003927255, -0.034997314, 0.022185797, -0.04127524, 0.017157773, 0.025395777, -0.04155931, -0.029060269, 0.026276391, -0.00020794745, 0.016646449, -0.022185797, 0.005706238, 0.0031016795, -0.0017239443, 0.04871785, 0.015382341, -0.008706718, -0.011071593, 0.022881765, -0.0011265115, 0.0035810461, -0.006047121, -0.018223032, 0.016717466, 0.008081766, 0.014537236, 0.02417428, 0.0067075817, -0.024785029, 0.022867562, -0.003605902, 0.02668829, 0.012548752, 0.04127524, 0.0050173705, 0.003909501, -0.000646701, -0.0077479845, 0.0030697216, -0.01874856, 0.007076871, -0.0020115643, -0.02762572, -0.0061003836, 0.0013848369, 0.0076911706, -0.020296738, -0.015282917, 0.00448119, 0.017001536, 0.014601151, 0.016206142, -0.016333973, 0.013784453, 0.011028983, -0.0058518234, 0.014210557, 0.050592706, -0.023918618, -0.040423032, 0.018024184, -0.0036112284, -0.017214587, -0.003359117, -0.0039947215, 0.011440882, 0.018720154, -0.012477735, -0.051530134, -0.013379654, -0.01443071, 0.032383878, 0.025921306, -0.034997314, -0.0036325336, 0.006402207, 0.009310365, -0.027100192, -0.042439923, 0.018535508, 0.017953167, 0.04116161, 0.01558119, -0.01747025, -0.0054079653, -0.018606525, 0.024160076, 0.0061252397, -0.004147409, -0.0005965451, -0.021390403, -0.0139762, 0.008408445, -0.006636564, -0.0074355085, -0.030310173, -0.006885125, -0.005443474, 0.008564684, -0.029912475, -0.0074639153, -0.000834453, -0.02742687, 0.018805373, 0.020921689, 0.014274472, -0.0044172746, 0.020396162, 0.0069383876, 0.014203454, 0.0007199376, 0.003153167, -0.007932629, 0.011376968, 0.036531284, -0.0028460172, 0.002212188, 0.0042077736, -0.02256929, 0.006277927, -0.0068425145, 0.015339731, -0.01952975, -0.016873704, -0.00025322096, 0.00785451, -0.013379654, 0.0020701536, -0.019643378, -0.018919002, -0.019231478, -0.049314395, -0.014615355, 0.0053724567, 0.0071691936, 0.04905873, -0.0043604607, 0.018066794, 0.022540882, -0.0035118042, -0.006927735, -0.01681689, -0.024373129, -0.0074426103, -0.058688674, 0.025296353, 0.0024944816, 0.00958023, -0.008571785, 0.002212188, -0.029571593, -0.0045983684, -0.048888292, 0.01570902, 0.025111709, -0.006050672, -0.011803071, -0.0023666506, -0.0019867083, -0.014686372, 0.010269098, -0.029287525, 0.04596238, 0.011810172, 0.016603839, 0.034855276, 0.016802687, -0.034798466, 0.030565834, -0.009679655, -0.0042929943, -0.007115931, 0.0030803743, 0.022356238, -0.011036084, -0.0076911706, -0.013919386, 0.02389021, 0.016234549, 0.0023542226, -0.024472553, 0.04724069, -0.0039698654, 0.025821881, -0.0180952, 0.01714357, 0.017995777, 0.0014851487, -0.033719003, 0.020296738, -0.005965451, 0.016703263, -0.03880384, -0.0038455853, -0.020509789, -0.04292284, -0.028037619, -0.0060542226, -0.0008806142, 0.0133938575, -0.02812284, -0.017086755, 0.027668329, 0.0370142, 0.026077542, 0.042496737, 0.0226119, -0.033207677, -0.0134648755, 0.0017132917, -0.014643762, -0.006572649, -0.015723225, 0.0063489443, -0.0010510556, 0.050592706, 0.009935317, -0.011646833, 0.0062637236, -0.010659693, 0.025892898, 0.0026400671, -0.032866795, -0.010780422, -0.0054257195, 0.04695662, 0.0037923225, -0.010340115, -0.008877159, 0.0022139635, 0.005464779, 0.014011708, 0.004243282, -0.01146929, 0.025665643, 0.018379271, 0.0018855087, -0.030651055, -0.033974662, 0.018279847, -0.028222265, -0.010531861, 0.008245106, 0.00760595, 0.03502572, 0.0076130517, 0.009459501, 0.02808023, -0.018308254, 0.009807485, 0.017200384, -0.009068905, -0.008720921, -0.011895393, 0.007733781, -0.00563167, -0.0242595, 0.029798849, -0.017171977, 0.01484261, 0.010205182, 0.029827256, 0.016561229, 0.032469098, 0.032042995, 0.0115829175, -0.013216315, 0.01422476, 0.025665643, -0.035593856, 0.015595393, 0.020921689, 0.059597697, 0.010119962, -0.0017550144, -0.011938004, 0.0056636278, 0.03851977, 0.036332436, 0.0146721685, -0.025196929, -0.021305183, -0.029457966, -0.013983301, -0.036559694, 0.04596238, 0.0011194098, -0.034315545, 0.008649904, 0.00706977, 0.025267946, 0.033037234, -0.013152399, -0.0029117083, 0.014288675, 0.01948714, 0.0051984643, 0.0018393474, 0.0012525672, -0.018436084, -0.04988253, -0.022029558, -0.014096929, -0.0020097888, -0.0027448176, 0.034855276, -0.021575049, -0.05451286, -0.029202303, -0.020580806, -0.002746593, 0.013500384, 0.031673703, -0.0151834935, -0.020779654, 0.0043853167, -0.0101412665, -0.022086373, 0.021702878, -0.004715547, 0.04221267, -0.0296, 0.024401534, -0.008614396, 0.013756046, -0.006402207, -0.023932822, 0.0076698656, -0.013479078], "content_hash": "49a8b51d502bfa2a8a28b13d02fe3d33"}, {"content": "experiments. ### Can I use my own model? Yes! Type any HuggingFace model ID into the model dropdown. The dashboard supports GPT-2, LLaMA, OPT, GPT-NeoX, BLOOM, Falcon, and MPT architectures. Unknown architectures may need manual configuration in the sidebar. ### What's the difference between the pipeline and the investigation panel?\n\nThe **pipeline** (5 stages) shows what happens during the model's forward pass -- how your input is processed step by step. The **investigation panel** (ablation + attribution) lets you run experiments to understand *why* the model made a specific prediction.\n\n### How do head categories get determined?\n\nThe dashboard automatically analyzes each attention head's pattern using heuristic rules (based on thresholds for attention distributions). For example, a head is classified as \"Previous-Token\" if more than 40% of each token's attention goes to the immediately preceding token. These categories are computed fresh for each analysis.\n\n### Can I save my results?\n\nCurrently, results are displayed in the browser and aren't saved between sessions. You can take screenshots or copy text from the chatbot (using the copy button on messages) to record your findings.", "source_file": "troubleshooting_and_faq.md", "chunk_index": 2, "embedding": [-0.012068988, -0.020482069, 0.066405654, 0.015807303, 0.010540699, -0.035809908, -0.061611023, 0.030415947, 0.025291685, 0.04138367, -0.02326895, -0.051931858, 0.0010038761, -0.021246213, -0.036439203, -0.012271262, -0.007487867, 0.0035285496, 0.0023579851, 0.055407967, -0.009147259, 0.0063828537, 0.019508159, 0.05190189, 0.0064877365, 0.0039967755, -0.028333278, 0.03428162, -0.009926387, 0.003925605, 0.007484121, -0.03173447, -0.05690629, 0.00961923, -0.036079608, 0.0138744665, -0.015672453, 0.04530927, -0.013537344, 0.025561383, -0.025771147, -0.011971597, -0.022984268, 0.043960784, -0.05588743, 0.019687958, 0.022789486, -0.010555682, -0.037188366, 0.06766424, -0.03428162, 0.039735515, -0.045369204, 0.007180711, -0.00043076772, 0.0065251947, -0.0150057, -0.012893065, -0.034161754, 0.002397316, -0.0131178135, -0.0059820525, 0.009911403, 0.030625712, -0.0017848767, -0.01806977, -0.066645384, 0.013327579, -0.026070813, 0.0064914823, 0.013305104, 0.009596756, 0.010533207, -0.017365558, -0.0071582356, -0.025216768, 0.023238983, 0.025201784, -0.021815576, -0.05148236, -0.019628026, -0.021216247, 0.00394808, -0.04018501, 0.0041878116, 0.009888928, -0.04863555, -0.03628937, -0.041503534, -0.04593857, -0.010353409, 0.02216019, -0.00448373, 0.04255236, 0.035270512, 0.0035416598, -0.0023748411, -0.05393961, 0.0034835998, 0.052441288, 0.05310055, -0.02094655, 1.7675524e-05, -0.0114846425, 0.040394776, -0.017380541, 0.0099039115, 0.022684602, -0.026175695, -0.009491873, -0.059872966, -0.036259405, -0.02620566, 0.016047034, -0.01110257, -0.021890491, -0.0318843, 0.025606332, -0.010076219, -0.04965441, -0.024048077, 0.026550274, 0.028483111, -0.020452103, 0.0367988, 0.018833915, 0.020032572, -0.040244944, -0.013125305, -0.015320348, -0.020017589, 0.02959187, 0.029696753, -0.029187324, -0.013769584, 0.0019646755, 0.012795675, -0.0030621965, -0.065147065, -0.026460376, 0.024152959, -0.029966451, -0.018099736, 0.0004991287, -0.07090062, 0.0030996548, -0.024167942, 0.058734242, -0.024197908, 0.038896453, 0.0184893, 0.019643009, -0.02439269, -0.0138744665, -0.039405882, -0.0054014525, -0.0024441385, 0.032303832, -0.03637927, -0.0694023, 0.013177747, 0.032273866, -0.0075140875, -0.045908604, -0.057026155, 0.06796391, -0.040125076, -0.009124784, -0.0368887, -0.016886095, -0.020047555, 0.016616397, -0.01486336, -0.018983746, 0.006959708, -0.014466304, -0.053969577, -0.025051953, -0.005360249, -0.049744308, -0.023448747, 0.025950946, -0.016466565, -0.0207218, -0.0511827, -0.05268102, -0.006184326, 0.016721278, 0.009589264, -0.006210547, -0.00436761, 0.045549005, 0.06221035, 0.023718445, -0.01979284, -0.016781213, 0.052111655, 0.007933618, -0.007637699, -0.023493698, 0.022265073, 0.009679164, 0.049594477, -0.028168464, 0.006146868, 0.04195303, 0.014728511, 0.010091202, -0.060382396, -0.03841699, 0.014361422, -0.013365037, 0.0022362464, 0.05190189, -0.08774177, -0.009027394, -0.067004986, 0.017665222, 0.018998729, 0.039885346, -0.0049369726, 0.026595226, 0.041143935, -0.012271262, -0.024707338, 0.048935212, 0.043241587, -0.032154, -0.016751247, 0.0020058793, 0.009828996, 0.019702941, -0.04018501, 0.0010076219, 0.027898766, -0.048186053, 0.008068467, -0.035090715, 0.01676623, -0.03530048, -0.0032419953, 0.0139418915, 0.0075927493, -0.0055063353, 0.061251424, 0.0034274128, -0.009319566, -0.056786425, 0.03586984, 0.013372528, 0.020377187, 0.0006536432, -0.017830038, -0.033712257, 0.010922771, -0.02638546, -0.047796488, 0.022969285, -0.04018501, -0.026745057, 0.025831081, 0.017530374, -0.048186053, 0.02494707, -0.029891534, -0.03601967, -0.024782255, -0.03925605, -0.07042116, -0.052021757, -0.0062217843, 0.0011377886, 0.0041915574, 0.010653073, -0.045878634, 0.034401484, 0.019628026, -0.02025732, -0.0055175726, -0.0037626624, -0.009574281, -0.01361226, 0.037068497, 0.015852252, -0.003268216, 0.01287059, 0.017620273, -0.0048171068, 0.016526498, 0.005090551, 0.034880947, -0.030400964, 0.048245985, -0.029157355, 0.009484381, 0.04905508, 0.0253666, 0.022055307, -0.009933879, 0.037877593, -0.0014355803, -0.04938471, 0.026175695, 0.007858702, 0.0141441645, 0.033322692, 0.011169994, 0.03925605, 0.023688478, -0.018923813, -0.023583597, -0.030925376, 0.018039804, 0.013417479, -0.012046513, -0.02211524, 0.033322692, -0.0144138625, 0.014593662, -0.019043678, -0.035899807, -0.046897493, 0.020512035, -0.0025808606, -0.03823719, -0.0048508192, 0.015207973, 0.005438911, -0.021440996, -0.011274877, 0.04102407, -0.039675582, -0.006461516, 0.007731344, -0.00042608546, -0.015462688, -0.008937494, 0.03428162, -0.0088775605, 0.04710726, -0.06179082, -0.01691606, -0.058464546, -0.009484381, -0.037428096, 0.02225009, -0.0017830038, 0.024662388, -0.012623367, -0.031045243, -0.009372007, -0.033742223, -0.007420442, -0.041143935, 0.027194554, 0.045608938, 0.00011781731, -0.029367121, 0.013432462, -0.02494707, -0.042911958, -0.01561252, 0.04339142, 0.009454415, -0.0018054786, 0.008053483, -0.08564412, -0.02755415, 0.0010319697, -0.0023149082, 0.021216247, -0.002378587, -0.0048021236, -0.011162503, -0.04803622, -0.0046448, -0.002901127, 0.019628026, -0.00076648564, 0.071080424, 0.049744308, 0.030610729, -0.030296082, -0.043601185, 0.034371518, 0.06370867, 0.05729585, -0.004060454, -0.021965409, 0.010705515, 0.0022362464, -0.032064103, -0.023763396, -0.023313899, -0.014676069, 0.043601185, 0.019867757, 0.03290316, 0.0047609196, 0.04297189, -0.019583076, -0.015133058, -0.029516954, -0.0042852024, 0.041084003, 0.033052996, 0.007442917, 0.0069784373, 0.052201558, -0.018848898, 0.022325005, -0.0465379, 0.029711736, -0.014578678, -0.03572001, 0.03122504, 0.0067611802, 0.002399189, 0.023583597, 0.016781213, -0.015155532, -0.041593432, -0.01941826, -0.0073043224, -0.045908604, 0.021261197, 0.03817726, -0.033562426, -0.005648676, -0.015552588, -0.018744014, 0.03536041, 0.0465379, -0.019822806, -0.01612195, -0.04399075, 0.035989705, 0.033981957, -0.023943193, 0.02355363, 0.03781766, -0.023388814, 0.013589785, 0.015672453, -0.05301065, -0.015447705, -0.060382396, -0.045399174, -0.029382104, -0.058104947, -0.0033787172, 0.022414904, -0.014623628, 0.050253738, -0.011080095, -0.06340901, 0.04321162, 0.0007458837, 0.050343636, 0.00863783, 0.0029236018, 0.013005439, 0.007030878, 0.07683398, -0.013305104, 0.02810853, -0.029247256, -0.025336634, -0.014166639, -0.041323736, -0.0017521009, -0.026010878, 0.026730074, -0.043181654, -0.008937494, -0.049414676, -0.00034695532, -0.014818409, -0.0064240578, -0.052111655, 0.019508159, 0.030086316, 0.005525064, 0.042911958, 0.044769876, 0.0921768, -0.018654115, -0.032453667, 0.038506888, 0.019388294, -0.0008769869, -0.02290935, 0.059363537, -0.019013712, 0.038896453, -0.037218332, -0.027584117, -0.03664897, -0.00878017, 0.0015779209, 0.028003648, 0.0162568, -0.007540308, 0.07018143, 0.0048508192, 0.015537605, 0.005202925, 0.004375102, -0.018159669, 0.041413635, -0.038446955, -0.0031745709, 0.010503241, 0.01788997, 0.0048058694, -0.026819972, -0.0023636038, 0.0061131557, 0.003933097, 0.00854793, 0.00510928, 0.01765024, -0.020182405, -0.012286245, -0.0061768345, -0.037607897, -0.028453145, -0.0694023, -0.03925605, -0.021006482, 0.0009224048, -0.06113156, 0.0141441645, -0.030805511, -0.052740954, -0.012653334, -0.0014636738, -0.03140484, -0.061101593, 0.016017068, 0.04204293, 0.005083059, 0.017065894, 0.013777075, 0.0071619814, -0.00901241, 0.0041353703, -0.01788997, 0.0044949674, 0.021710694, 0.010106185, -0.030925376, 0.028932609, -0.01198658, -0.009746588, 0.0072219144, -0.010136152, 0.010061236, -0.0035641347, -0.014855868, 0.0098364875, 0.024812222, 0.022444872, -0.02503697, 0.015088107, 0.03604964, 0.013904433, -0.023463732, -0.022085274, -0.011304843, -0.008922511, 0.0025602586, 0.007562783, -0.013125305, 0.0063116834, 0.018324485, -0.00743168, 0.035390377, -0.026400443, -0.01087033, -0.032273866, -0.026819972, -0.027793882, 0.017065894, -0.008068467, 0.016886095, 0.016226834, -0.0069185044, -0.010877822, -0.01078043, 0.018294519, -0.053969577, 0.011282369, 0.025141852, -0.029966451, -0.015432722, 0.03512068, -0.004547409, -0.02216019, -0.019672975, 0.02490212, -0.0146835605, -0.0013690921, 0.004453764, -0.01997264, -0.0162568, -0.05648676, 0.005453894, 0.0027194554, -0.022055307, 0.005880916, -0.013657209, -0.02857301, -0.036678936, -0.029007524, 0.041143935, -0.04785642, 0.030131266, 0.03038598, -0.02890264, -0.02058695, -0.004094166, -0.010488258, 0.0029816618, 0.02523175, 0.0071432525, 0.008390606, -0.012443569, -0.017815055, 0.021830559, -0.025156835, 0.00039986483, 0.0414436, 0.006551415, -0.00627048, 0.008278232, 0.025216768, -0.015432722, -0.0054314192, -0.018848898, 0.01826455, -0.013762092, -0.020616919, -0.038536854, 0.00063819176, -0.001779258, 0.0111999605, 0.011035145, -0.030685645, -0.01696101, -0.027943715, -0.022774503, 0.0326035, 0.02443764, 0.006776164, -0.027344385, -0.004888277, -0.0070383702, -0.015477671, 0.034671184, -0.055407967, 0.04668773, -0.0047272076, -0.032813262, -0.04390085, 0.008293215, 0.0018803948, -0.0023860787, 0.027703984, 0.0014983225, -0.03098531, 0.01528289, 0.031764437, 0.023179049, 0.009866454, -0.0011864841, -0.0043863393, -0.07611478, -0.013005439, 0.058644343, -0.016975993, 0.0018101609, -0.07018143, 0.005513827, 0.04000521, 0.016901078, 0.006135631, 0.0075328164, -0.0064090746, 0.02917234, -0.0045174425, -0.0095667895, -0.017680205, -7.9656915e-05, -0.011904173, 7.555994e-05, -0.014428846, 0.033292726, 0.016990978, -0.033472527, -0.029427055, -0.031524707, 0.006330413, -0.034581285, 0.012952998, -0.0068398425, 0.0009991939, 0.03994528, 0.024287809, -0.01941826, -0.043780982, 0.0065926192, 0.011514609, 0.038716655, -0.03107521, 0.020062538, -0.019028695, 0.02875281, 0.004892023, -0.04282206, -0.0066488064, 0.026145728, -0.03919612, -0.005974561, 0.04168333, -0.018429367, -0.031704504, 0.0019983877, -0.031974204, 0.0052366373, 0.018938797, 0.00961923, -0.017320609, -0.04734699, 0.004229015, 0.009469398, -0.005749813, 0.0023336373, -0.0032831992, -0.020197388, 0.0066637895, 0.015342822, -0.015432722, -0.0117543405, 0.037937526, 0.012593401, -0.0010741099, 0.019867757, -0.0028262108, -0.04770659, 0.0060457313, 0.015178007, -0.026969805, 0.00031066782, 0.004079183, -0.021381062, 0.018384418, -0.00878017, -0.007948601, 0.031554673, 0.027029738, 0.038357057, -0.009304583, 0.019268427, -0.02207029, -0.015462688, 0.031045243, 0.024377707, 0.010098694, -0.0070271324, -0.009102309, -0.025306668, 0.02216019, -0.00743168, 0.073417805, -0.03646917, 0.044799842, 0.016931044, 0.009754079, 0.007956092, 0.0064690076, -0.00067096757, 0.0060382397, -0.025636299, -0.0035173122, -0.032273866, -0.018833915, -0.014054265, -0.03404189, -0.041653365, -0.03637927, -0.009971336, 0.004229015, -0.019298393, -0.013320087, -0.0019553108, 0.0005014698, -0.0974509, -0.023628546, 0.011724373, -0.027718967, 0.015537605, -0.020781733, 0.0050755674, -0.005611218, 0.008922511, -0.022969285, -0.008982443, -0.022280056, 0.014721019, -0.0135223605, 0.00026407934, -0.03586984, 0.009694147, -0.0020527018, 0.031015277, -0.015642487, -0.04060454, -0.0025490213, -0.009911403, 0.015732387, -0.04668773, -0.0018691573, -0.03577994, 0.020616919, 0.035749976, 0.005708609, -0.027299436, 0.01890883, -0.0026052082, 0.005764796, -0.018384418, 0.056516726, -0.030655678, -0.0020077522, 0.010046252, -0.010465783, -0.021650761, -0.018803949, 0.02207029, -0.013792058, 0.019163545, 0.0069484706, -0.018803949, 0.007236898, -0.027299436, 0.01937331, -0.019073647, -0.008165858, 0.026700107, 0.061161526, 0.022429889, -0.01937331, -0.009342041, -0.0073380345, 0.014938275, -0.027464252, -0.00743168, 0.02290935, 0.009754079, -0.012278753, -0.003462998, 0.005105534, 0.014084232, 0.033202827, -0.02592098, -0.032363765, -0.069522165, 0.017635256, -0.002859923, 0.006210547, 0.007427934, 0.0020564476, 0.015095599, 0.014129181, -0.008263249, -0.016376665, 0.02852806, -0.012518485, -0.056276996, 0.019837791, -0.00599329, 0.039136183, 0.042102862, -0.01579232, 0.010885313, 0.008233283, 0.034521352, 0.00032354402, -0.0016931044, 0.012563434, -0.034671184, -0.009784046, -0.027718967, 0.020347219, 0.05301065, -0.012660826, 0.01862415, 0.011042637, -0.004618579, 0.0032944367, -0.002361731, -0.011649458, -0.0040267417, -0.010787922, -0.019013712, -0.008360639, 0.014878343, 0.014518745, 0.00046213885, -0.0068398425, 0.0071469983, 0.007701378, 0.060232565, -0.002942331, 0.016990978, 0.0021388554, 0.021650761, 0.0077388356, -0.009229667, 0.035090715, -0.0007819371, 0.010053745, 0.010143643, -0.009304583, -5.393376e-05, -0.014983226, 0.011612, -0.008592879, -0.0023842058, 0.0059670694, -0.007941109, -0.0030640697, 0.025291685, -0.006195564, -0.010375883, 0.0056674047, -0.023433764, 0.025381584, 0.004888277, -0.027014755, -0.027539168, 0.0054426566, -0.007971075, -0.027763916, 0.0022868149, 0.031764437, 0.03506075, -0.0039855377, -0.01714081, 0.0059445947, 0.00845803, 0.029606853, 0.02373343, -0.042372562, -0.034641217, 0.03781766, 0.021201264, 0.043541253, 0.02156086, -0.031195074, 0.021770626, 0.0017904954, -0.0033318948, 0.012473535, 0.022864401, 0.045339238, -0.035150647, 0.021366078, 0.043601185, 0.0050118887, 0.038776588, 0.044020716, 0.010001303, 0.020152438, -0.016721278, -0.0025565128, -0.01788997, 0.035989705, -0.0004759515, 0.017590307, 0.02169571, 0.010698023, -0.019178528, 0.039106216, -0.0048508192, 0.028003648, 0.0208117, 0.010832872, -0.002339256, 0.055677664, -0.023538647, -0.014061756, -0.0006007337, 0.0036690172, -0.0601127, -0.008997426, -0.011582033, 0.034731116, -0.0022531026, -0.028333278, 0.009049868, 0.050613336, -0.012878082, 0.015043158, -0.0020733038, 0.0068286047, -0.013724634, 0.011035145, -0.018519266, 0.0013578548, -0.01714081, 0.009207192, 0.02625061, 0.010211068, -0.025066936, 0.0012820022, -0.030191198, -0.029741703, -0.04761669, -0.007173219, -0.0072743557, 0.034161754, 0.03907625, 0.009252141, -0.0032064102, 0.020467086, -0.0025359108, -0.0129829645, 0.026310543, 0.015462688, -0.011252402, 0.004094166, 0.015507638, -0.016751247, 0.012833132, 0.000248862, 0.02684994, 0.024602456, -0.025651282, -0.01645158, 0.015357805, 0.006907267, 0.010585649, 0.0008006661, 0.008697762, -0.008787662, -0.004757174, 0.026175695, 0.024197908, -0.0013232061, 0.026040845, 0.0130503895, 0.03404189, 0.02810853, 0.023718445, -0.00669001, -0.0025471484, 0.04432038, -0.004217778, -0.019523142, 0.005274095, -0.0002341129, -0.026745057, -0.014323963, 0.015335331, 0.0161669, -0.0092970915, 0.028857691, -0.027389336, 0.012406111, -0.024962053, 0.021141332, -0.005304062, -0.012458552, -0.00989642, 0.003888147, 0.0020189895, -0.0021238723, -0.014511254, -0.023658512, -0.012735741, -0.032064103, -0.036619004, -0.006480245, 0.010623107, 0.0022605942, -0.0044350345, 0.014563695, 0.01156705, -0.026520308, -0.013529852, 0.022504805, -0.008705254, -0.016316732, 0.024542524, -0.031045243, 0.006030748, -0.020841666, -0.002339256, -0.06610599, 0.0081358915, -0.0076789027, 0.021470962, -0.049834207, -0.0049819225, 0.028183447, -0.0064315493, -0.0010488258, -0.004229015, 0.027868798, 0.004176574, -0.003498583, 0.020676851, -0.017365558, -0.03107521, -0.01797987, -0.016331716, 0.030041367, -0.018309502, -0.0101885935, -0.0053415196, -0.042432494, 0.025576366, -0.009446924, -0.004989414, -0.008398098, 0.012256279, 0.001914107, 0.01806977, 0.002181932, -0.039735515, -0.008555422, -0.033981957, -0.024647405, -0.015537605, -0.01528289, 0.0017904954, 0.042492427, -0.017725155, 0.018788964, 0.018998729, -0.008487997, 0.0033187843, -0.012503501, 0.028048597, -0.018039804, 0.06179082, -0.014316471, -0.03754796, -0.014443829, -0.0013559819, 0.021261197, 0.021440996, -0.027958699, -0.019538125, -0.003461125, 0.012720758, 0.03293313, -0.018968763, 0.03043093, 0.0011087586, 0.0010188593, -0.0067986385, -0.009229667, 0.028947592, -0.03158464, 0.009491873, 0.039915312, 0.0052703493, -0.0063379044, 0.021306146, 0.021545878, -0.006330413, -0.014114198, 0.0024478845, 0.02443764, -0.021141332, -0.035000816, -0.009694147, 0.0252767, -0.015837269, -0.0018626022, -0.0031314942, 0.052141625, -0.0048433277, -0.018309502, 0.015642487, 0.008338165, 0.017485423, -0.025007002, 0.0029329665, 0.0013719016, 0.005334028, 0.0012885574, 0.007776294, -0.027898766, -0.0032457411, 0.016062018, 0.014743494, -0.012878082, 0.017005961, -0.035839874, -0.009079834, 0.03586984, 0.02490212, 0.014241556, -0.02448259, -0.00047876086, -0.029606853, 0.018309502, 0.0014177876, -0.052381355, -0.024317775, -0.012016547, 0.005004397, 0.020556984, 0.008900036, -0.0076489365, -0.00040735645, -0.017904954, 0.05199179, 0.029157355, -0.0015189245, -0.017620273, 0.021006482, 0.01862415, -0.003522931, -0.0007355827, -0.016706295, -0.009694147, 0.008075958, 0.013297613, 0.021590827, 0.0014618009, -0.028423179, 0.007701378, 0.0029442038, 0.006768672, 0.028737826, 0.0551982, 0.00036334322, 0.03934595, -0.022010358, -0.014915801, 0.02220514, -0.018039804, -0.0082707405, 0.00094160205, -0.02388326, 0.0046522915, -0.013222696, 0.014286505, -0.031944238, 0.0150356665, 0.0066375686, 0.018938797, -0.01078043, 0.017320609, -0.027748933, 0.030625712, 0.001989023, -0.001768957, 9.2650174e-05, 0.039016318, -0.03985538, -0.050014004, 0.024212891, 0.0015994593, 0.00047618564, 0.013065373, 0.015987102, 0.012488518, 0.031344905, -0.027464252, -0.016901078, -0.0070271324, -0.008855086, 0.029891534, 0.02192046, -0.022564737, -0.013110322, -0.010061236, -0.017170776, -0.049774274, -0.050523434, 0.02388326, 0.023104133, 0.045189407, 0.0097016385, -0.03823719, 0.014810918, 0.0036615257, 0.02021237, 0.010915279, 0.02452754, 0.010046252, -0.022744536, -0.020961532, 0.029681768, -0.025246734, 0.011559558, -0.021935442, -0.0019496921, -0.01705091, -0.015013192, -0.003477981, -0.014511254, 0.00680613, -0.026280576, 0.013859483, 0.0033038012, 0.007405459, -0.011417218, 0.025771147, 0.008817628, -0.0010619361, 0.00743168, -0.031195074, -0.0031202566, 0.006379108, 0.011649458, -0.016391648, 0.0032345038, 0.009132275, -0.024347741, -0.013357545, -0.006210547, 0.0026407933, -0.009873945, -0.021440996, 0.0010197958, -9.47572e-05, -0.013200222, 0.009342041, -0.041773234, -0.011042637, -0.030685645, -0.036858734, 0.0066787726, -0.011342301, 0.017290642, 0.07072082, -0.0036072114, 0.046238232, 0.031165108, -0.012578418, -0.014368913, 0.012368652, -0.039166152, 0.0005501653, -0.04129377, 0.023074167, 0.0051317546, -0.012503501, -0.012690792, 0.03577994, -0.00028983178, 0.0050868047, -0.041054036, -0.00092849176, 0.032273866, -0.009274617, -0.0041503534, -0.010915279, 0.026595226, -0.021725677, 0.015133058, -0.01477346, 0.03892642, -0.014721019, -0.014765969, 0.016316732, 0.031704504, -0.020152438, 0.007971075, -0.0043975767, 0.009169734, -0.024737304, -0.00034555062, 0.031105176, -0.009641705, -0.0068735546, -0.01676623, -0.013065373, -0.017230708, 0.0056823883, -0.0049557015, 0.031165108, -0.014428846, 0.04725709, -0.002764405, 0.018384418, 0.021036448, -0.0028355753, -0.0129829645, 0.0008657495, 0.009109801, 0.034910914, -0.024692355, 0.00062742253, -0.020407153, -0.033442557, -0.015927168, -0.010757956, -0.029741703, 0.00074494723, -0.008233283, -0.0061431224, 0.010683039, 0.005734829, 0.035690043, 0.032213934, -0.015672453, -0.0033038012, -0.027898766, 0.00924465, -0.0029086187, -0.007506596, -0.012705775, -0.021171298, 0.04240253, 0.039106216, 0.004097912, 0.0037589166, -0.022384938, -0.013087847, 0.008113417, -0.012248786, -0.0059783068, -0.018684082, -0.01170939, 0.008630338, 0.019987622, 0.003805739, 0.00789616, 0.0036390508, -0.0042252694, 0.018429367, 0.0139418915, -0.0053527574, 0.024197908, 0.025576366, 0.0014187242, -0.022924334, -0.031794403, 0.0323338, 0.015912184, -0.038806554, 0.0056524216, -0.002241865, 0.05217159, 0.0011256147, 0.0126833, 0.014084232, -0.010151135, -0.0050718216, 0.015972119, -0.0071619814, -0.015327839, -0.0040304875, 0.023673495, 0.0010516351, 0.003539787, 0.0026333018, -0.018699065, -0.009064851, 0.021815576, 0.009304583, 0.008165858, 0.025441516, 0.02634051, -0.010518224, 0.014706035, 0.026370477, 0.019852774, -0.025666265, 0.013005439, 0.015627503, 0.037727762, 0.005124263, -0.0044387802, -0.031135142, 0.0026464122, 0.0060270024, 0.026475359, 0.0037476793, 0.003933097, -0.035270512, -0.018414384, -0.008360639, -0.013657209, 0.02857301, 0.020407153, -0.044050682, 0.014189114, 0.01263835, 0.010248526, 0.038626753, -0.015537605, 0.010952737, -0.028453145, 0.003423667, -0.015133058, 0.007933618, 0.014855868, -0.04051464, -0.029442038, -0.0061393767, 0.0021538385, 0.032034136, -0.012301228, 0.027569134, -0.010765447, -0.029337155, -0.0114846425, -0.0024609948, -0.01477346, -0.012855607, 0.041233838, -0.023523664, -0.021321129, 0.0063903457, 0.016151916, -0.009004918, 0.013215205, -0.033951987, 0.022864401, -0.027748933, 0.013866975, -0.00459985, 0.0122263115, -0.008817628, -0.029861568, 0.0301013, -0.0324237], "content_hash": "641cc2ed4a5b47aa3552073d7a4473fe"}, {"content": "# What Is a Large Language Model (LLM)?\n\n## The Big Idea\n\nA Large Language Model is a computer program that has learned to read and write text by studying enormous amounts of human writing. Think of it like an incredibly well-read assistant that has absorbed millions of books, articles, and websites, and can now predict what word comes next in a sentence.\n\n## How It Works (Simply)\n\nAt its core, an LLM does one thing: **next-token prediction**. Given some text like \"The cat sat on the\", the model predicts the most likely next piece of text (called a \"token\") -- perhaps \"mat\" or \"floor.\"\n\nThis might sound simple, but to do it well, the model has to understand grammar, facts, context, and even some reasoning. All of that understanding is encoded in the model's **parameters** -- billions of numbers that were learned during training.\n\n## What Is a Neural Network?\n\nAn LLM is built on a type of computer program called a **neural network**. A neural network is loosely inspired by the brain: it's made of layers of simple processing units that pass information forward, transforming it step by step. Each layer takes input numbers, multiplies and adds them, and passes the result to the next layer.\n\nWhen you stack many layers together and train them on lots of data, the network learns complex patterns -- like how words relate to each other.\n\n## What Makes It \"Large\"?\n\nThe \"large\" in LLM refers to two things:\n\n- **Many parameters**: Modern LLMs have billions of learnable numbers (GPT-2 has 124 million; larger models have tens or hundreds of billions).\n- **Massive training data**: They train on huge text datasets -- sometimes trillions of words from the internet, books, and code.\n\n## How Does This Connect to the Dashboard?\n\nThe Transformer Explanation Dashboard lets you look inside an LLM as it makes a prediction. When you enter a prompt and click \"Analyze,\" you can see:", "source_file": "what_is_an_llm.md", "chunk_index": 0, "embedding": [-0.008591564, -0.007622969, 0.011082237, -0.017233444, 0.052983407, -0.012906214, -0.0023821779, 0.017761769, -0.04354904, 0.09077119, -0.0078053665, -0.028428894, -0.044983063, -0.015346571, -0.00204883, -0.028126994, -0.003610218, 0.010553912, 0.021950629, 0.052580874, 0.02012665, 0.0061103255, 0.012994269, 0.027950887, 0.033334766, -0.015874896, -0.015497521, 0.044832114, 0.008220479, -0.0015598154, 0.010384094, -0.021510357, -0.066015415, 0.011094816, -0.014516347, 0.015170462, 0.020768188, 0.02880627, 0.013321327, 0.040328775, -0.049737986, -0.025862746, -0.01296911, 0.01774919, -0.056153353, 0.044203155, 0.006516003, -0.002750118, -0.042316284, 0.049888935, -0.019208372, 0.027850254, -0.031548526, -0.015862316, -0.020315338, -0.017183129, -0.01790014, 0.026919395, -0.020026017, -0.045612022, 0.009635634, -0.009308576, 0.010038167, 0.03801421, -0.017095074, -0.028605003, -0.041083526, 0.014050918, -0.018541677, -0.011585403, 0.03285676, -0.0071386714, 0.019497693, -0.012805581, -0.009906086, -0.00641537, -0.04251755, 0.028001202, -0.031246625, -0.0097236885, 0.010962735, -0.026894236, -0.005993968, -0.03756136, 0.021522937, -0.018403307, 0.008346271, -0.010308619, -0.028101835, -0.04699573, 0.022340583, -0.028705636, -0.009899796, 0.022755694, 0.03134726, 0.008245637, 0.0042737685, -0.004298927, 0.0016384352, 0.04088226, -0.009302286, 1.3869011e-05, 0.038567696, -0.058971085, -0.0087550925, -0.019120319, 0.016944123, -0.022579586, -0.01923353, -0.027825095, -0.11180355, 0.0020598369, 0.008817988, -0.019812172, 0.019623484, -0.036806613, -0.0016384352, 0.034164988, 0.0001027953, -0.031875584, -0.0048241066, -0.024378406, 0.018830998, -0.012214361, 0.026466545, 0.01832783, 0.0022265108, 0.035171323, 0.029711967, 0.026667811, -0.013686122, 0.0021180154, 0.00925197, -0.06767586, -0.042115018, -0.0005200695, -0.03637892, 0.032278117, -0.015195621, 0.0079185795, 0.044983063, -0.020415971, -0.007994054, 0.011868434, -0.04830396, -0.00544363, -0.0024607976, 0.039901085, -0.0596252, 0.00424861, 0.011069658, -0.010648256, -0.040404253, 0.009333734, -0.022088999, -0.01535915, 0.0065222927, 0.050014727, 0.0135603305, -0.04830396, 0.032378748, 0.055599872, 0.013396801, 0.036957562, -0.013510014, 0.07834299, -0.022894066, -0.013145219, -0.03134726, -0.054643854, -0.02566148, 0.016604487, 0.0076103895, -0.04060552, 0.013384223, -0.031850424, -0.041637007, -0.05192676, -0.0012885773, -0.07185214, -0.044328947, 0.011811828, -0.052128024, -0.021849995, -0.02459225, 0.012503682, -0.0090129655, 0.03386309, -0.015233358, -0.019170634, 0.0014992782, 0.052077707, 0.091777526, 0.055398606, 0.028403735, 0.011925041, 0.07134897, -0.009358892, 0.04712152, 0.019573167, 0.05710937, 0.06465686, 0.027699303, -0.07975185, 0.028504368, 0.024642568, -0.0077927876, 0.015094988, -0.025711795, -0.034642998, 0.00999414, -0.020516604, 0.058820136, -0.010031877, -0.04103321, -0.01790014, -0.065109715, -0.0047140387, 0.00603485, 0.022164473, 0.008113556, 0.031699475, 0.013635806, 0.024441302, -0.067625545, 0.025988538, 0.051624857, -0.02624012, 0.027120661, 0.0003203754, 0.049587034, -0.021585833, -0.008365139, -0.051624857, -0.018742943, -0.051247485, 0.0102708815, 0.008365139, 0.022101577, 0.0018271225, 0.010453279, 0.015522679, 0.030139659, 0.016315166, 0.010277171, 0.012906214, -0.023296598, -0.009415499, 0.06651858, 0.046693828, 0.019937962, 0.0077361814, -0.045461074, -0.0052329293, -0.02624012, 0.032001372, -0.04709636, 0.025711795, -0.034391414, -0.021887733, -0.0033460557, 0.0153339915, -0.04966251, 0.017409552, -0.0055851457, -0.024277773, -0.00014072932, -0.010296039, -0.060480583, -0.016478695, -0.005912204, -0.026542021, -0.026516862, 0.0027123806, -0.039448235, -0.084230036, -0.0070631965, 0.018629732, 0.040026877, 0.046341613, 0.0011808684, -0.008409166, 0.042416915, 0.029737126, -0.008100977, 0.0014906301, -0.005368155, -0.01684349, -0.009295996, -0.030818934, 0.023334336, -0.014742771, 0.0140131805, -0.023686552, -0.025837587, 0.01640322, -0.01298169, 0.042693656, -0.00589648, 0.037485886, 0.022692798, -0.034441732, -0.020026017, -0.026441388, -0.01788756, 0.060027733, -0.0050631105, -0.011679747, 0.004487614, -0.004100805, 0.008547537, 0.013459697, 0.042869765, 0.013245852, -0.02223995, -0.016818332, 0.01537173, -0.033938564, 0.011006762, -0.020051176, 0.0030740313, -0.017547924, 0.027472878, -0.017560503, -0.012170334, 0.02400103, 0.019321585, 0.005324128, -0.027246453, -0.030919567, 0.075726524, 0.04754921, -0.012516261, 0.024026189, -0.028655319, -0.07421702, -0.02236574, 0.02161099, 0.014994355, 0.04415284, -0.010182827, -0.0041982937, -0.024780938, 0.01670512, -0.0064059356, 0.030919567, 0.0017123377, 0.018994527, 0.059826467, 0.005934217, -0.015409467, 0.0041668457, 0.00910102, -0.021912891, 0.009182785, 0.03074346, -0.0060914564, -0.013082323, 0.052882776, -0.005267522, -0.058266655, 0.0022249382, 0.0094343675, 0.014101234, -0.04968767, 0.01566105, -0.025334422, -0.0032092573, 0.012264677, -0.023070173, 0.040932577, 0.007176409, 0.02460483, -0.01567363, -0.047347944, 0.011277214, -0.0015716084, 0.014881142, -0.05328531, 0.049436085, 0.021044929, 0.034970056, -0.0012131024, 0.007396544, 0.0055222497, 0.04400189, -0.001789385, -0.011371558, 0.00042140175, -0.00020303545, -0.025321841, -0.02354818, -0.012692369, 0.0073273587, 0.010855812, 0.017283762, 0.05489544, 0.04193891, 0.039146334, 0.029057851, -0.013132639, -0.008717355, -0.02863016, -0.0014489616, 0.04322198, 0.013774176, 0.027623829, 0.024416143, 0.028982377, 0.010075904, 0.044328947, 0.007352517, 0.008182742, -0.042593025, 0.04100805, 0.023523023, 0.021938048, 0.046618354, 0.020151809, 0.022315424, -0.022579586, -0.03746073, -0.025032522, -0.022441216, -0.0012964393, 0.011132553, 0.03117115, -0.0075978106, -0.036479555, -0.025812428, 0.013409381, 0.04098289, 0.031523366, -0.017371815, -0.007692154, -0.03217748, 0.019107739, 0.05036694, 0.018214619, -0.033787616, 0.04221565, 0.011302372, 0.008327401, 0.0010676559, 0.018692628, 0.022114158, -0.055901773, -0.021963207, 0.007824236, -0.031800106, 0.026667811, -0.010088484, -0.000518104, 0.040756468, -0.027850254, -0.055901773, 0.05187644, -0.022780852, 0.07270753, -0.027976044, 0.04566234, 0.02742256, 0.0054813675, -0.053536892, -0.008767672, 0.001410438, 0.008189031, -0.02147262, 0.039574027, 0.0041416874, -0.007050617, -0.038416743, 0.03190074, -0.0005845377, 0.004858699, 0.019698959, 0.024944467, -0.0325297, 0.0033366212, -0.029183643, -0.0010778764, 0.0077298917, 0.002937233, -0.018705206, -0.008843147, 0.04787627, -0.032907072, -0.0013475422, 0.027372245, 0.0031903887, -0.026315596, -0.0069562737, 0.037234303, -0.012799292, 0.005710937, -0.0023727433, -0.024416143, -0.080959454, 0.013006848, -0.023938134, 0.015749104, -0.02549795, -0.032303274, 0.06893378, 0.0041982937, 0.025133155, 0.07295911, 0.018755522, -0.026969712, 0.02744772, -0.010836943, 0.00068123994, 0.013887389, 0.02281859, -0.013359064, 0.025812428, 0.0033932275, 0.010402963, 0.0142396055, 0.023120489, 0.010245723, 0.012598026, -0.027674144, -0.015598154, 0.016692542, -0.0051480196, -0.03159884, -0.0028444617, -0.022164473, 0.0028916337, 0.00701288, -0.02176194, 0.009667082, 0.011164001, -0.05771317, -0.007968896, -0.003474992, -0.029460385, -0.06948726, -0.012774133, 0.005789557, 0.051021058, 0.055247657, 0.009937534, 0.004188859, 0.012849608, 0.0115287965, -0.037963893, 0.0007842318, 0.016629646, 0.043649673, -0.012654631, 0.017686294, -0.032554857, -0.016918965, -0.011604272, -0.012289836, 0.0016950413, 0.014126393, -0.014252184, -0.0012469089, -0.007220436, 0.0049750563, -0.0025708652, -0.0062707094, -0.010094773, 0.0067801652, 0.016969282, 0.012742685, 0.017396973, -0.007975185, -0.024466459, 0.012704948, 0.028957218, 0.002327144, -0.04548623, 0.017975615, 0.009667082, 0.00045009796, 0.0417628, -0.029636493, -0.0393476, -0.006755007, 0.0023019856, -0.030793775, 0.032881916, -0.026617495, -0.012648342, -0.031473048, 0.011969067, 0.03401404, -0.029762285, -0.01745987, 0.01684349, -0.053335622, -0.002243807, 0.022126736, -0.04802722, -0.033183817, 0.013346485, -0.0048555546, -0.02983776, -0.010862102, 0.008402877, 0.02923396, -0.025586005, -0.040001716, 0.021837415, -0.009579028, -0.010384094, -0.038416743, -0.035448063, -0.008289664, -0.056153353, -0.0408571, 0.03358635, -0.012956531, -0.008327401, 0.02893206, -0.02593822, 0.0014175137, -0.010956446, 5.6016557e-05, -0.021598412, 0.008082108, 0.0033051735, -0.022554427, 0.021070087, -0.04203954, 0.028479211, -0.022390898, 6.825175e-05, 0.07673285, -0.012843319, 0.0031353547, 0.03791358, 0.00879912, -0.030995041, 0.001797247, -0.018604573, 0.010421831, 0.011371558, -0.015522679, -0.0037674573, -0.006905957, -0.0003001308, -0.02223995, -0.004371257, -0.003088183, 0.0011171864, -0.010541333, -0.022806011, 0.04203954, 0.028001202, 0.003430965, -0.02162357, 0.009201653, -0.017535344, 0.005500236, 0.038391586, -0.053637523, -0.0055599874, 0.00031585476, -0.01342196, -0.020667555, 0.027774777, 0.029561019, 0.00804437, 0.019409638, 0.01296911, -0.07250626, 0.022403479, -0.00081685896, 0.008899753, -0.027472878, -0.0067675863, -0.0064059356, -0.037183985, 0.0056574754, 0.02236574, 0.00075435627, 0.020831084, -0.023673972, -0.008623011, 0.00026632432, 0.012201782, 0.03265549, 0.014189289, 0.007616679, 0.023195965, -0.010220565, -0.052429926, -0.02461741, -0.05144875, -0.0043177954, -0.040680993, -0.013522593, 0.08739998, -0.009163915, -0.09766457, -0.010050746, 0.012327573, -0.053084042, 0.0043460988, -0.010912418, -0.023120489, -0.04324714, 0.004239176, 0.015220779, -0.007415413, -0.022038681, 0.014881142, 0.020529183, 0.011170291, -0.028605003, -0.014189289, 0.0020897123, -0.022894066, -0.008189031, -0.018063668, -0.066719845, 0.005431051, -0.02507026, -0.020013439, 0.042718817, -0.015296254, -0.014629559, 0.0075286254, -0.016818332, 0.017963035, -0.009459525, 0.013824493, 0.011403006, -0.039121177, -0.01567363, 0.0008530241, -0.032102007, 0.003311463, -0.055952087, -0.008591564, -0.03758652, -0.0023758882, -0.03922181, 0.03308318, 0.032605175, 0.00910102, 0.0086921975, 0.025284104, -0.022629902, -0.036026705, -0.052178342, 0.0102708815, -0.0008420173, -0.013673543, 0.018126564, -0.013296168, 0.017485028, 0.0043115057, -0.0041039498, -0.017623398, -0.0032296984, 0.009849479, -0.009157626, 0.036303446, 0.0087236455, -0.023762027, -0.0034561232, -0.026290437, 0.0024340667, -0.010874681, -0.03668082, -0.021183299, 0.024114244, -0.0052738115, 0.0023051305, -0.023095332, 0.03620281, 0.022265106, -0.0002789035, 0.019032264, -0.009880927, -0.021095246, 0.034718473, -0.053889107, 0.014390555, 0.009371472, -0.04369999, -0.03610218, -0.01566105, -0.020101491, -0.022252528, -0.0148182465, 0.024516776, -0.020680133, 0.013057165, 0.020680133, 0.023145648, -0.088154726, -0.04531012, 0.042693656, -0.0044121393, 0.00054208306, -0.019309005, 0.009050703, 0.0152710965, 0.0075663626, -0.03773747, 0.0049310294, -0.03640408, 0.0050190836, 0.03144789, -0.010201696, -0.057310637, -0.013648385, -0.007591521, 0.012277257, 0.016290007, -0.022843748, 0.020013439, -0.035699647, 0.011723774, -0.010396673, -0.008472062, -0.01894421, 0.025887905, 0.014063497, 0.016491273, -0.013522593, 0.02835342, -0.012874767, 0.01042812, 0.0055599874, -0.00032017883, -0.0048932917, -0.04739826, -0.038819276, -0.016667383, -0.0063021574, -0.045989398, 0.03029061, -0.02699487, 0.027850254, -0.009667082, 0.008390297, -0.003965579, -0.008031791, -0.01342196, -0.023824923, 0.010101063, 0.02863016, -0.0021148706, 0.0076669957, 0.0006379991, -0.004607116, -0.0035567565, 0.037511043, 0.053084042, -0.005902769, -0.0024089084, -0.025422476, -0.0073902546, -0.020982033, 0.026818762, 0.009572738, 0.027976044, -0.026265278, -0.025271526, 0.01402576, 0.017258603, -0.0001171434, 0.017170548, 0.030667983, 0.021384567, 0.0024183427, 0.04058036, 0.0140131805, 0.0049561877, 0.022151895, -0.014076076, -0.02460483, 0.0049939253, 0.02354818, 0.03907086, 0.020818504, -0.013660964, -0.017635977, 0.03897023, 0.025736954, 0.027347086, -0.01282445, -0.0063807773, -0.016918965, -0.0044058496, 0.007585231, 0.037183985, 0.04445474, -0.030064184, 0.04012751, 0.040957734, -0.043624517, -0.028454052, -0.029762285, -0.021070087, -0.02400103, -0.014138972, 0.0049719117, 0.0039718687, 0.021145562, 0.034341097, 0.00894378, 0.04531012, -0.019472534, 0.027498037, 0.040731307, 0.030391242, 0.030944725, -0.007126092, -0.021095246, 0.01073631, -0.02520863, -0.012855898, -0.01968638, 0.006723559, 0.0018349845, 0.029183643, 0.00047958034, 0.00076890097, -0.03280644, -0.0070443274, -0.0039152624, -0.02893206, -0.02611433, 0.016327744, -0.0149063, 0.035121005, -0.0063902116, 0.023623656, 0.008880884, 0.023296598, 0.023007277, -0.026592337, 0.021359408, 0.013761598, -0.0298126, -0.06651858, -0.024365826, 0.016076162, 0.047046047, 0.015208201, -0.0079185795, 0.058367286, -0.016013267, -0.008025502, -0.001617994, -0.0626442, -0.035976388, 0.015346571, 0.0011706478, 0.033787616, -0.0022878342, 0.00070954306, 0.034114674, -0.002231228, -0.02759867, 0.0058839004, -0.029686809, 0.014956617, -0.010761469, 0.00037167477, 0.023535602, 0.0067675863, 0.02340981, 0.011711195, 0.0048398306, 0.022026103, 0.020403393, 0.013975443, -0.018969368, 0.02714582, 0.0043209405, 0.05368784, 0.004890147, -0.009635634, 0.0045379307, -0.020252442, -0.021875152, 0.037812945, 5.208557e-05, 0.01850394, 0.03285676, 0.015107567, 0.0051039928, -0.036932405, 0.0046008267, 0.04485727, -0.0014489616, -0.010572781, -0.02071787, 0.03401404, -0.0053178384, -0.0066040573, -0.02938491, 0.03356119, 0.005547408, 0.0068682195, -0.005874466, 0.002734394, -0.01491888, -0.0011148277, -0.017761769, 0.022177054, 0.01118287, -0.013811914, 0.020013439, 0.004204583, 0.01431508, -0.036429238, -0.02283117, -0.024944467, 0.024944467, 0.03371214, -0.0006588333, 0.020101491, 0.018994527, 0.0051071374, -0.012956531, -0.007251884, -0.004635419, -0.012101148, 0.023925556, 0.0013247425, -0.003953, -0.005484512, 0.008906043, -0.009830611, 0.036127336, -0.0007799077, 0.013258431, -0.02596338, -0.049134184, -0.0023585919, 0.0049467534, 0.028026361, 0.016679961, 0.0023727433, -0.031523366, -0.023950715, 0.008107266, 0.027648985, 0.015434625, -0.03788842, 0.03474363, -0.0049153054, 0.023824923, 0.036051862, 0.013899968, -0.008824279, -0.014478609, 0.051272642, 0.012327573, 0.0018035366, 0.005056821, -0.0127867125, -0.0027108083, -0.0023853225, 0.0075286254, 0.0032988838, -0.06878283, 0.04442958, -0.015824579, 0.02266764, 0.0033775035, 0.0024356393, 0.00477379, -0.0028617582, 0.004371257, 0.03801421, -0.028227627, 0.012906214, -0.0006737711, -0.027196137, -0.0064185145, -0.0015810428, -0.008264506, 0.0034120963, 0.017208286, -0.026013697, -0.019472534, 0.026642654, 0.005387024, -0.005773833, -0.011252056, 0.0015944081, 0.0028601857, -0.008377718, 0.0025032521, -0.018453622, 0.03295739, 0.016793175, -0.0047454867, -0.02938491, -0.008616722, -0.019044843, -0.007585231, -0.04578813, -0.0115287965, 0.0070631965, -0.008025502, 0.0022611034, 0.038567696, 0.023573339, -0.007339938, -0.007874552, -0.010314909, -0.005833584, -0.00079327304, -0.010667125, -0.008924912, -0.012346442, -0.0028869165, -0.0107551785, 0.005314694, 0.0058052805, 0.032278117, -0.0061952346, 0.015006933, 0.026466545, 0.04216533, 0.0018585704, -0.003758023, 0.01656675, -0.04576297, -0.020705292, -0.014554084, -0.035699647, 0.026189804, -0.027724462, 0.037989054, 0.03620281, 0.0022894065, -0.005726661, 0.0033743586, -0.009176495, 0.010641966, -0.02117072, 0.010321198, 0.011730064, 0.06043027, -0.025422476, -0.01894421, -0.0013239562, -0.008824279, 0.035749964, -0.016239692, -0.013220694, -0.032026533, -0.005993968, -0.013975443, 0.021271354, -0.011245766, -0.012516261, 0.044278633, -0.0019010251, -0.006308447, -0.0014937748, 0.009924955, 0.012906214, 0.014440872, -0.005462499, -0.029007535, -0.033737298, -0.011566535, 0.018428463, 0.0071638296, -0.009289707, 0.01908258, 0.032026533, -0.013799335, -0.016302587, -0.01103192, 0.033787616, -0.008327401, -0.008409166, 0.015006933, -0.0023129925, -0.032303274, -0.0063021574, 0.005726661, -0.0094658155, -0.034517206, -0.014855984, 0.0054499195, 0.023837501, 0.009591606, -0.014377976, -0.014667297, -0.01236531, -0.024491617, -0.029057851, 0.010340067, -0.00925197, 0.014755351, -0.029862918, 0.017635977, -0.0024435013, -0.00044852556, 0.036856927, -0.012252099, 0.04996441, -0.032001372, 0.017560503, 0.011063368, -0.029586175, -0.019359322, 0.024554513, -0.0072393045, 0.023636235, -0.0140131805, -0.0044372976, 0.007855684, 0.007830525, 0.01237789, 0.0059562307, 0.012371601, -0.020831084, -0.0034026618, 0.019623484, -0.031749792, 0.013698702, -0.013032006, 0.03029061, 0.007679575, -0.02130909, 0.006060009, -0.018994527, -0.029183643, 0.008899753, -0.008924912, 0.020214705, 0.031422734, 0.039045703, -0.003905828, -0.013899968, -0.025837587, -0.0008758238, 0.02103235, -0.024202297, -0.008107266, -0.006399646, -0.022277687, -0.013182956, 0.018390726, 0.027699303, -0.03655503, 0.009138757, -0.012516261, 0.004937319, -0.028504368, -0.023535602, 0.03532227, -0.0126735, 0.04354904, -0.0011958061, 0.00056724134, 0.03761168, -0.018151723, -0.038265795, 0.014390555, -0.0067298487, 0.018755522, 0.017095074, 0.005028518, -0.017573081, -0.0077739186, 0.0030818933, -0.0077487603, 0.010623097, 0.0058681765, 0.010685993, 0.018768102, -0.017837243, 0.005934217, -0.00044931174, 0.004663722, -0.012591735, -0.008616722, 0.018277515, 0.02641623, 0.023321757, 0.008836857, -0.013623226, 0.029963551, 0.001745358, 0.026064012, 0.003096045, -0.016290007, 0.0063650534, -0.023309177, 0.004679446, -0.0058839004, -0.008824279, 0.008931201, -0.02908301, 0.005534829, 0.0052109156, -0.0044687456, -0.013610647, 0.024265192, -0.0012862188, -0.010170248, 0.010604229, 0.026642654, 0.010698573, -0.035422906, 0.025284104, -0.0023979016, -0.013195535, -0.015698787, -0.0010645111, -0.0029844048, 0.0024655147, 0.0057077925, -0.041561533, 0.0035976388, 0.008585274, -0.024554513, -0.008088398, -0.013057165, 0.020642396, -0.013258431, -0.030089343, -0.044278633, 8.9528214e-05, 0.011063368, -0.0087550925, -0.0061386283, 0.0016918966, 0.021887733, -0.025133155, -0.038039368, -0.010761469, -0.009283418, 0.022541849, 0.0056889234, 0.018025931, 0.043599356, -0.011566535, 0.0018868735, -0.02669297, -0.023950715, -0.0075223357, -0.0007225153, 0.055549555, -0.010069615, 0.013208115, 0.007352517, -0.018151723, -0.008365139, -0.029611334, -0.0155604165, 0.025636321, 0.012195492, -0.015082409, -0.02863016, -0.006251841, -0.020013439, -0.028076677, 0.018264934, -0.022478953, 0.0071009337, -0.008484641, -0.025057679, -0.012321284, -0.019019684, -0.02265506, 0.0012382608, -0.0028869165, 0.0060946015, -0.018277515, -0.02074303, 0.020176968, 0.0036511002, -0.009484684, 0.011541376, 0.013736439, 0.0048146723, 0.038039368, -0.014730193, 0.05036694, 0.01832783, 0.03864317, 0.015447204, 0.025095418, 0.020680133, 0.04324714, -0.008119846, 0.01222694, 0.012302415, -0.0033523452, 0.008811699, 0.0055536977, 0.01446603, -0.035900913, 0.00939034, 0.008113556, 0.01984991, 0.002467087, -0.037183985, -0.0034970054, -0.018617151, 0.01417671, 0.0064656865, 0.02790057, -0.007446861, -0.028126994, -0.02369913, 0.04473148, 0.005028518, -0.014403135, -0.0098369, -0.0011903027, -0.012283546, 0.017120233, -0.0170196, -0.024554513, 0.007968896, 0.0155604165, -0.045108855, 0.0024875281, -0.0028586134, -0.029032694, 0.018617151, 0.0114973495, 0.018755522, 0.020176968, 0.0026762155, 0.0068996674, -0.016717698, 0.015786842, 0.008107266, -0.028906902, 0.013497435, 0.035900913, -0.024680305, 0.015107567, -0.008987808, 0.057310637, -0.031120833, -0.012258388, 0.023346914, 0.01971154, 0.015774263, 0.029032694, 0.017824665, 0.02832826, -0.029460385, 0.00079405925, -0.0020315337, -0.0126735, -0.021422304, 0.012799292, 0.007427992, -0.029284276, -0.010491016, -0.017485028, 0.0021164431, 0.062241666, 0.011962778, 0.0041857143, 0.0039404207, 0.024793517, 0.054694172, 0.0014866991, 0.015032092, 0.011723774, 0.015484942, 0.009377761, 0.0075034667, -0.00045324274, 0.03341024, 0.0038492219, 0.026290437, 0.021799678, -0.004607116, 0.022353161, 0.038416743, -0.014881142, -0.03029061, 0.0051354407, -0.036127336, -0.0064782654, -0.011019341, 0.024944467, -0.0067675863, -0.020617237, -0.020831084, 0.019057423, 0.023145648, 0.03622797, -0.0097865835, -0.028504368, 0.0008656032, 0.016906386, 0.014491188, -0.01563589, -0.04905871, -0.023082752, -0.019044843, 0.008623011, 0.025032522, 0.01698186, 0.02669297, 0.0064027905, -0.0048461203, -0.046593197, -0.020680133, -0.018541677, 0.014868563, 0.024328088, 0.019321585, 0.0055411183, 0.0002535486, -0.0066292156, 0.0005479795, -0.008761383, -0.0015574568, -0.018868735, 0.031221466, -0.008251927, 0.0307183, -0.0019985135, 0.022277687, -0.004833541, -0.017334078, 0.0062486962, 0.013132639], "content_hash": "08aa475fc59a563dd5d1f9e81d489071"}, {"content": "train on huge text datasets -- sometimes trillions of words from the internet, books, and code. ## How Does This Connect to the Dashboard? The Transformer Explanation Dashboard lets you look inside an LLM as it makes a prediction. When you enter a prompt and click \"Analyze,\" you can see:\n\n- How the model breaks your text into tokens (Stage 1)\n- How those tokens become number vectors (Stage 2)\n- How the model figures out which words relate to each other (Stage 3: Attention)\n- How knowledge is retrieved from the model's memory (Stage 4: MLP)\n- What the model predicts as the next token (Stage 5: Output)\n\nThis step-by-step view helps you understand what happens inside the \"black box\" of an LLM.", "source_file": "what_is_an_llm.md", "chunk_index": 1, "embedding": [0.012544138, 0.009709375, -0.008504289, -0.03119488, 0.047479156, -0.0050357627, 0.01578476, 0.050476264, -0.0053666933, 0.08586709, 0.00989045, -0.013399563, -0.04255891, -0.043058425, -0.014211278, -0.034741458, 0.015010507, 0.020829888, 0.0131498035, 0.07567693, 0.01820742, -0.0072554946, 0.023464844, 0.018781865, 0.0137742, 0.02300279, -0.028872123, 0.019331334, 0.007399106, -0.010908218, 0.0022603176, -0.014923091, -0.065286964, -0.015360169, 0.0373889, -0.003031448, -0.002202561, 0.059942123, -0.010308797, 0.042808667, -0.0038244324, -0.012974972, -0.0549969, 0.05339844, -0.028997002, 0.018619522, -0.019381287, -0.013974008, -0.03861272, 0.06998243, -0.0036464792, 0.034366816, -0.03619006, 0.015347681, -0.02056764, -0.009453372, -0.038862478, 0.015185338, -0.028172798, -0.021579165, 0.023314988, -0.005622696, 0.013237219, 0.037613682, -0.038512815, -0.016121933, -0.046804808, 0.021129599, -0.029471545, 0.0046673687, 0.03798832, -0.024264073, 0.028722268, -0.003468526, 0.0007141542, -0.011482663, -0.00046244412, 0.028122846, -0.04528128, 0.0020667545, 0.029021978, -0.04727935, -0.028072895, -0.016296765, 0.023627188, -0.019543631, 0.0042739986, -0.002789494, -0.0020370956, -0.07722544, -0.00068800757, -0.008110919, -0.051250517, 0.023090206, 0.028272701, 0.030595459, -0.016271789, -0.010327528, -0.022403369, 0.03628996, 0.0053385952, 0.0002290757, 0.045356207, -0.041360065, -0.014635868, -0.020592617, -0.0021931948, -0.032094013, -0.036240008, 0.0005092739, -0.08991319, -0.008335701, -0.015272753, 0.00083903363, 0.072879635, -0.012419259, -0.003655845, -0.017907709, -0.010271333, -0.01979339, -0.03004599, 0.005663282, -0.003908726, 0.009347225, 0.0015274314, 0.019081576, 0.01990578, -0.0020776815, 0.042284172, 0.01517285, -0.017945174, -0.010021574, 0.019493679, -0.05000172, -0.041509923, -0.008398142, -0.017682927, 0.028322654, -0.010371236, 0.020967256, 0.028922075, -0.0015360169, 0.009596984, 0.0104087, -0.046754856, -0.0013330879, 0.00075591076, 0.04460693, -0.04540616, -0.00030985707, -0.01347449, -0.008947611, -0.006712269, -0.005244936, -0.02809787, -0.00018566058, -0.0014384548, 0.0118073495, 0.030070966, -0.03841291, 0.0057350877, 0.03519102, 0.02020549, 0.01168247, -0.034966238, 0.039661705, -0.027823135, -0.020442761, -0.03441677, -0.049677037, -0.021579165, -0.020043148, 0.00943464, -0.02779816, -0.012999948, -0.041834608, -0.043932583, -0.047629014, -0.004558099, -0.037713587, -0.04583075, 0.005207472, -0.0122881355, -0.021117112, -0.024363976, 0.0068184165, -0.025750138, 0.045056496, -0.01043992, 0.0063438746, -0.01886928, 0.034266915, 0.04388263, 0.006549926, -0.0011941595, 0.023377428, 0.066935375, 0.023302501, 0.033068072, 0.03526595, 0.032968167, 0.06978262, 0.012038376, -0.058293715, -0.022753032, 0.05135042, -0.016159398, 0.0033342806, -0.01092695, -0.015385145, 0.04995177, -0.0058256253, 0.03586537, 0.0038275544, -0.053997863, -0.011626274, -0.06648581, 0.015085435, 0.005844357, 0.028697291, 0.018769378, 0.033467688, 0.016309254, 0.012000913, -0.06009198, 0.030595459, 0.024963398, -0.044631906, 0.024201633, 0.009253565, 0.040685717, -0.0028503728, -0.031020049, -0.0225782, 0.008554241, -0.056895066, -0.005388547, -0.024676174, 0.025600282, 0.0048390776, 0.004177217, 0.022415858, 0.022141121, 0.019868316, 0.028697291, 0.028272701, -0.010414944, -0.025200669, 0.059992075, 0.04126016, 0.01858206, 0.0012370868, -0.047229398, -0.027948016, -0.015385145, 0.040560838, -0.045730844, 0.0020511446, -0.015110411, -0.011089293, 0.022865422, 0.022740543, -0.039237116, 0.0180201, -0.010670947, -0.019331334, 0.014061423, -0.014960555, -0.05025148, -0.045306254, -0.024476368, -0.015460073, -0.017795319, 0.027373571, -0.05949256, -0.06898339, 0.024551295, 0.003015838, 0.023702115, 0.054946948, -0.0029736913, -0.010827046, 0.028672315, 0.019631045, -0.013037412, 0.013199755, 0.01595959, -0.012219451, 0.0008999124, -0.015160362, 0.016284278, -0.020705009, 0.036339913, -0.01383664, -0.014423573, -0.0062127514, -0.019206455, 0.018769378, 0.011950961, 0.01125788, 0.019705974, -0.018831817, -0.01298746, 0.004177217, -0.010514848, 0.057094872, 0.01723336, -0.024888469, 0.031144928, -0.008928879, -0.017145945, 0.005479085, 0.06903335, 0.022865422, 0.0021604141, -0.03144464, 0.037239045, -0.025575306, 0.030870194, -0.03077029, -0.020193003, -0.04643017, 0.0021042184, -0.005859967, -0.015847199, -0.019144015, -0.028522462, 0.010190161, -0.017370729, -0.022753032, 0.034491695, 0.0035777956, -0.010652215, -0.00731169, -0.043982532, -0.017982638, -0.005454109, 0.020467738, -0.022328442, 0.046779834, -0.07058185, -0.008916391, -0.06488735, 0.007961064, -0.0338673, 0.0521996, -0.010040306, 0.00861668, 0.039062284, 0.0042240466, -0.0052980096, -0.025750138, 0.0077175484, -0.05449738, 0.026799126, 0.03186923, -0.021841412, -0.0026880295, 0.057894103, 0.0028987636, -0.0303457, 0.0025725162, 0.0048453216, 0.025412964, -0.054847043, 0.047254376, -0.015210314, 0.0063563627, 0.017170921, -0.051799987, 0.044831716, -0.015797248, 0.022028731, -0.0048796637, -0.030545507, -0.0036277473, 0.008941367, 0.011975937, -0.048628047, 0.0746779, 0.017620487, 0.004411366, -0.005023275, -0.0024835395, 0.023414893, 0.04485669, -0.0021182674, -0.013799177, 0.0029471545, 0.023265038, -0.0035715515, -0.017483119, -0.015634904, -0.018756889, -0.0036433572, 0.00024331975, 0.05254926, 0.036664598, 0.047778867, 0.028422557, -0.03429189, -0.023602212, -0.04290857, -0.023577236, 0.047479156, 0.036864407, 0.011226661, -0.016034517, 0.037488803, -0.022690592, 0.07148098, -0.030195845, 0.018282348, -0.021279454, 0.016896186, 0.014298694, 0.005791283, 0.030420627, 0.012213208, 0.051600177, -0.061990146, 0.0003797115, -0.051600177, -0.0070806635, -0.027948016, 0.022503272, 0.046380218, -0.020255443, -0.012669018, -0.007748768, 0.017383216, 0.054547332, 0.008179602, -0.0028019822, 0.0006848856, -0.046929687, 0.019206455, 0.022078682, 0.013449514, -0.01523529, 0.033942226, -0.011626274, -0.010777094, -0.03658967, -0.041010402, 0.031294785, -0.033242904, -0.03047058, -0.018219909, -0.032493625, 0.031244833, 0.0010872314, 0.011008121, 0.04096045, -0.057994004, -0.0848181, 0.040086295, -0.0019855828, 0.05764434, 0.008935123, -0.0064375345, 0.013212243, 0.008229554, -0.02221605, -0.039661705, 0.019468702, 0.010215137, -0.04613046, 0.06728503, 0.032069035, -0.018607035, -0.044532005, 0.017433168, 0.010571043, 0.023989338, 0.020405298, 0.02694898, -0.032693435, -0.003708919, -0.03726402, 0.0115950545, -0.020542666, -0.012850093, -0.027298642, 0.012406771, 0.09865475, -0.06728503, -0.0023133913, -0.010983146, -0.010340016, -0.005226204, -0.02797299, 0.006755977, -0.013087364, 0.0018388495, -0.02215361, -0.052599214, -0.058843184, -0.00441761, -0.020854864, -0.00018809963, -0.0014329914, -0.023814507, 0.05339844, 0.030021014, 0.029721303, 0.05564627, 0.016921163, -0.013561905, 0.005416645, -0.02336494, 0.0005084934, 0.01256287, 0.0020027538, -0.014660845, 0.011570078, 0.010090258, 0.011975937, 0.019393776, 0.012431746, 0.010489872, 0.02039281, -0.010265089, -0.012163256, 0.00907249, -0.008448093, -0.047653988, -0.044082437, -0.037838466, 0.008079699, 0.0023789532, -0.012862581, 0.032069035, 0.0094471285, -0.0577942, -0.01047114, 0.004117899, -0.024576271, -0.04558099, 0.0064812424, 0.022927862, 0.043507993, 0.03404213, 0.012338087, 0.0057538194, 0.010827046, -0.0037869685, -0.024513831, -0.0048983954, 0.011694958, 0.052099697, -0.033068072, 0.011669982, -0.02622468, -0.020367835, -0.02707386, -0.0068995883, -0.0055321585, 0.007180567, -0.047454182, 0.0007157152, 0.010933193, 0.019506166, -0.020904817, -0.012444234, 0.017195897, 0.007867404, -0.008672876, -0.018120004, -0.0050794706, 0.014173815, -0.027473474, 0.006103482, 0.008117163, -0.017008578, -0.017820295, 0.024513831, -0.0032000353, -0.0031297905, 0.011251636, -0.029446568, -0.03144464, -0.02020549, -0.022253513, -0.023165133, 0.035815418, -0.015160362, 0.0035184778, -0.029696327, -0.004361414, 0.04198446, -0.021778971, -0.015647393, -0.0026302729, -0.048178483, -0.015897151, 0.03611513, -0.014136351, -0.027173763, 0.012512919, 0.016671402, -0.018981673, -0.017583024, -0.01886928, 0.03004599, 0.0022431468, -0.03916219, 0.016121933, -0.022503272, -0.0053073755, -0.023077717, -0.048328336, -0.024339, -0.018482154, -0.025100764, 0.030370677, -0.0050139087, -0.0028503728, 0.037538756, 0.0012768921, -0.01778283, -0.006930808, 0.0067871967, -0.02251576, -0.0028940807, 0.00067161716, -0.008267018, 0.00012819654, -0.06898339, 0.022653127, -0.067784555, 0.026899029, 0.058293715, -0.022428345, -0.00044332197, 0.04468186, 0.015772272, -0.01899416, 0.007786232, -0.025999896, 0.0031204247, 0.005672648, -0.019818366, -0.0022041218, 0.007661353, 0.017982638, 0.0010138649, 0.0024663687, -0.0078361835, -0.01935631, -0.042808667, -0.03628996, 0.037488803, 0.021266967, 0.01220072, 0.0010443042, 0.0014087961, -0.013749225, 0.01839474, 0.018369764, -0.032968167, -0.019231431, 0.012194476, -0.029171834, -0.041434992, 0.04927742, 0.02694898, 0.027423522, 0.02471364, 0.024938421, -0.061890244, 0.020005684, -0.010377481, 0.02792304, -0.007917356, -0.018644499, -0.023227572, -0.046929687, -0.017795319, 0.025338035, -0.015085435, -0.008960099, -0.021466773, 0.0012370868, -0.00081952126, 0.02427656, 0.029996037, 0.0040523373, 0.011988425, 0.025512867, -0.03916219, -0.02767328, -0.0055415244, -0.023539772, -0.006755977, -0.033667494, -0.008554241, 0.07917356, 0.007367886, -0.062239904, 0.0078361835, -0.02051769, -0.03768861, -0.024801053, 0.0022540737, -0.0337674, 0.00069971505, 0.0062627033, 0.03628996, -0.032393724, 0.00015492854, 0.014910603, 0.023265038, -0.009778059, -0.042359103, 0.056345597, 0.007979795, -0.014610893, -0.013861616, -0.015435097, -0.035940297, 0.02949652, -0.03673953, -0.029346665, 0.023040254, -0.041509923, -0.026674245, -0.0032000353, -0.0059629926, 0.016496573, 0.022028731, 0.026174728, 0.0025131984, -0.0789238, 0.0076800846, 0.0032125232, -0.0155100245, 9.785474e-05, -0.039062284, -0.028422557, -0.049726985, 0.008354434, -0.023689628, 0.030145893, 0.011182953, -0.011432711, -0.008629168, 0.019131528, -0.026299607, -0.031644445, -0.024226608, 0.009690643, -0.0006345436, 0.0037963346, -0.007929844, -0.005360449, 0.032069035, -0.0016156276, 0.015584952, -0.015672367, 0.0075177415, 0.005360449, 0.00023590504, 0.02724869, -0.0006973735, -0.017932685, -0.00086479, -0.008572972, 0.021104623, -0.00391497, -0.0017139701, -0.011769886, 0.050651096, -0.013212243, 0.022003755, -0.021966292, 0.015310218, 0.026674245, 0.01353693, -0.007367886, -0.03853779, -0.007917356, 0.0041897045, -0.032918215, -0.0035996495, -0.0017795318, -0.032493625, -0.013736737, 0.0016437254, -0.014086399, -0.019006649, 0.0071118833, 0.04041098, -0.031469617, -0.0014400159, 0.0024772957, 0.029846182, -0.09390933, -0.050750997, 0.022790495, 0.0021323161, 0.016221838, -0.03186923, 0.015047971, 0.008129651, 0.024026802, -0.021641605, -0.02234093, -0.03361754, -0.019618558, 0.020954767, 0.036664598, -0.0536482, 0.0024913445, -0.017108481, 0.01028382, 0.0032031573, -0.024363976, -0.01334961, -0.01334961, 0.003893116, -0.041235186, -0.015897151, -0.015310218, 0.016921163, -0.006368851, 0.032543577, -0.0016109446, -0.0061940197, -0.014473526, -0.003053302, -0.010152698, 0.0037526267, -0.008142139, -0.023165133, -0.025387987, -0.0053073755, -0.00523557, -0.021816436, 0.0288971, -0.016534036, 0.023190109, 0.00701198, -0.014835676, -0.013486978, 0.017495608, -0.014261231, -0.022028731, -0.0068995883, 0.02264064, -0.004074191, 0.01292502, 0.017682927, 0.010770851, -0.006315777, 0.021079646, 0.021753997, -0.0035122337, -0.013911568, 0.0014509428, -0.015909638, 0.0004924932, 0.032243866, 0.008466825, 0.029096907, -0.032443676, -0.024563784, 0.0010372797, -0.017707903, -0.0020901693, -0.0027145664, 0.007386618, 0.015285241, -0.0039711655, 0.033817347, 0.005032641, -0.012693994, 0.04468186, -0.015285241, -0.015547489, 0.008098431, 0.04240905, 0.02463871, 0.014698308, -0.02627463, -0.012862581, 0.03429189, 0.023152646, 0.022977814, -0.015222802, -0.014261231, -0.008897659, 0.015247778, 0.012194476, 0.03514107, 0.054597285, -0.023277525, 0.035390828, 0.041859582, -0.03926209, -0.02809787, -0.030920146, -0.0043270723, 0.0039056041, -0.011838569, -0.004545611, -0.004142875, -0.002603736, -0.017133458, 0.00649373, 0.025163203, -0.029346665, 0.042234223, 0.034741458, 0.03416701, 0.031070001, -0.044956595, -0.014560941, 0.024039289, -0.045630943, -0.011638762, -0.01693365, 0.019094065, 0.0024944665, 0.0009888889, -0.0024991494, 0.015747296, -0.012812629, 0.005894309, 0.008572972, -0.025038324, -0.033242904, 0.019081576, 0.0006513242, 0.06353865, -0.017170921, 0.01772039, -0.0051668864, 0.041360065, 0.041834608, -0.033167977, -0.0036152594, -0.0024460757, -0.010171429, -0.027423522, -0.017795319, 0.022615664, 0.03301812, 0.024576271, -0.012300624, 0.03534088, 0.0057975273, 0.012544138, -0.014461038, -0.08082196, -0.029121881, 0.029846182, 0.03356759, 0.022553224, 0.008473069, -0.017820295, 0.028197775, 0.0010255723, -0.022990303, 0.020255443, -0.037239045, 0.019318847, -0.01174491, 0.018045077, 0.020667545, -0.021391846, 0.01881933, 0.012700237, 0.00569138, 0.03841291, 0.011070561, 0.014248743, -0.02270308, 0.06039169, 0.00605353, 0.030120917, 0.0075052534, 0.007917356, 0.006755977, 0.005703868, -0.034017157, 0.03628996, -0.00152509, 0.011482663, 0.019431239, 0.061041065, -0.019506166, -0.04643017, 0.008291994, 0.049052637, -0.006968272, -0.028447533, -0.02749845, 0.042933546, 0.020967256, -0.005226204, -0.008878927, 0.025350524, 0.01911904, 0.0009545471, -0.002589687, -0.001963729, -0.01803259, 0.017095994, -0.013249707, 0.030070966, 0.02099223, -0.037663635, 0.041385043, -0.02264064, 0.024189144, -0.015460073, -0.030645411, -0.033167977, 0.027873088, 0.0098155225, -0.01086451, 0.0021853899, 0.020779936, 0.021354381, -0.0068371487, 0.008023503, 0.010290065, -0.04498157, 0.020767448, 0.011389004, -0.001738946, 0.020492714, 0.01784527, -0.006278313, 0.040336054, 0.0002136609, 0.011794861, -0.016658915, -0.033367783, 0.005697624, 0.015834711, 0.026074825, 0.025400475, 0.018856794, -0.018969186, -0.016209349, 0.008004772, 0.02804792, 0.015609928, -0.01886928, 0.02026793, -0.00026458828, 0.022440832, 0.030520532, -0.0042771203, 0.015934614, 0.0013861617, 0.03716412, 0.0015828467, -0.02355226, -0.003481014, -0.007317934, -0.031119952, -0.0118073495, 0.028847147, 0.016396668, -0.061190918, 0.043932583, -0.0018903624, 0.0024757346, 0.016409157, 0.0006158117, 0.00012985509, -0.022191074, -0.0024538806, 0.0027676402, -0.015485049, -0.030270772, 0.01735824, -0.018681962, 0.02003066, 0.011513883, -0.021092135, 0.017495608, 0.01608447, -0.046704907, -0.024388952, 0.015047971, 0.0147607485, -0.008060967, 0.00034966238, -0.001971534, -0.0072617386, 0.0050732265, 0.018607035, -0.03484136, 0.011620031, 0.016484084, -0.00066537317, -0.032118987, -0.0015906517, -0.016783794, 0.0015430414, -0.040885523, -0.010321285, 0.0040523373, -0.021142086, -0.0024788566, 0.010240112, 0.022827959, -0.00955952, -0.008985074, 0.0010005964, -0.008878927, -0.006531194, -0.016746331, -0.013099852, 0.007886136, -0.004573709, 0.005419767, 0.004046093, -0.022603177, 0.018594546, -0.008341946, 0.0250633, 0.022927862, 0.011657494, 0.024613734, -0.023589723, 0.004302096, -0.02882217, -0.021042183, -0.02355226, -0.03089517, 0.0027629572, -0.017170921, 0.039187163, 0.03841291, -0.013624345, -0.0029502765, 0.0055415244, 0.0035715515, 0.013137315, -0.017395703, 0.04031108, 0.004807858, 0.0577942, -0.026024872, -0.0048515657, -0.007667597, -0.0066373413, 0.014635868, -0.024538808, 0.0022743666, -0.038462862, 0.0061440677, -0.006353241, 0.009971622, -0.016833747, -0.007886136, 0.035240974, -4.226876e-05, -0.0017155311, -0.014723284, 0.020442761, -0.0014993337, 0.022303466, -0.02015554, -0.009153662, -0.00962196, 0.013387075, 0.014473526, -0.021641605, -0.0091786375, 0.020655056, 0.013224731, -0.0042365347, -0.05135042, 0.006150312, 0.003015838, -0.020080611, 0.012569115, 0.0042365347, 0.02015554, -0.03866267, -0.005435377, 0.031419665, 0.00435517, -0.012000913, -0.018045077, -0.046654955, 0.004089801, 0.0064375345, -0.0046985885, -0.03331783, -0.010933193, -0.017607998, -0.0033342806, 0.009946646, -0.00048781026, 0.018257372, -0.05479709, 0.015834711, 0.009896695, 0.0015126021, 0.017258337, -0.008498045, 0.03047058, -0.0059317728, 0.015210314, 0.0006267386, -0.02907193, -0.024438903, 0.02634956, -0.033517636, 0.013936544, 0.002417978, -0.0015586513, -0.0037682366, -0.003680821, 0.0035372097, 0.026924005, 0.03534088, -0.00053464004, -0.0021042184, 0.0021182674, -0.03981156, 0.019206455, -0.0038556522, 0.008167114, 0.00737413, -0.02639951, 0.006512462, -0.027848111, -0.013561905, 0.0044488297, -0.0033904763, 0.031469617, 0.042259198, 0.026474439, 0.0038337985, -0.016983602, -0.032243866, -0.023439867, 0.050026696, -0.03149459, -0.01971846, -0.013499466, -0.027023908, -0.0066623176, -0.00082966767, 0.017957661, -0.047778867, 0.0037463827, -0.022203563, -0.002399246, -0.04011127, -0.008510533, 0.009753083, -0.0066623176, 0.036639623, -0.032019086, 0.0075614494, 0.0444321, -0.015922127, -0.019493679, 0.021891363, -0.0028534948, 0.023889434, 0.019281384, 0.016708868, -0.023027766, -0.009309761, 0.0057538194, -0.016721355, 0.0060285544, 0.000861668, 0.013611858, 0.024126705, -0.032843288, -0.010583531, 0.0046111727, 0.009759327, -0.018120004, -0.0014938702, 0.02779816, 0.0429835, 0.029346665, 0.0112703685, -0.027048884, 0.065237015, 0.008460581, 0.020505201, 0.0022400247, -0.017033553, 0.0048952736, -0.011557591, -0.004130387, 0.006450022, -0.014136351, 0.015872175, -0.041959487, -0.01730829, -0.008079699, 0.003265597, -0.009703131, 0.004820346, -0.007174323, -0.02003066, 0.015909638, 0.009303518, 0.002602175, -0.016534036, 0.0053573274, 0.01773288, -0.022740543, -0.026474439, 0.008011015, 0.0032187672, 9.2927854e-05, 0.010333773, -0.039486874, 0.018419715, 0.008629168, -0.041484945, 0.0035372097, 0.021104623, 0.023302501, 0.012581602, -0.01132032, -0.011769886, 0.020292906, -0.034316868, -0.010770851, -0.011195441, -0.006749733, 0.0074677896, -0.037963346, -0.012007156, -0.0025803212, -0.012157012, 0.034491695, -0.019656021, 0.034341842, 0.023577236, 0.006456266, 0.007911111, -0.00786116, -0.023414893, 0.012800141, -0.016296765, 0.043907605, -0.018319812, -0.0046704905, 0.020317882, -0.012825117, -0.022927862, -0.022303466, -0.02597492, -0.011226661, 0.037239045, 0.018781865, -0.0036090154, -0.0078112083, -0.0046767346, -0.02822275, 0.023727091, -0.017283313, -0.003986776, -0.0027770062, -0.0095033245, 0.00037112605, -0.008085943, -0.03089517, 0.024351489, -0.00786116, 0.012556626, -0.017970148, -0.028147822, 0.010477384, 0.010777094, 0.023327477, -0.003062668, 0.012238184, 0.020667545, 0.031719375, -0.024776079, 0.034941263, 0.023889434, 0.031070001, 0.015472561, 0.03229382, 0.028247725, 0.03976161, -0.016783794, 0.014960555, 0.020342859, -0.016621452, -0.004767272, -0.008135894, -0.02100472, 0.004121021, 0.014136351, 0.019406263, 0.010852022, -0.014211278, -0.010290065, -0.018057564, -0.023452356, -0.0018591424, -0.0021416822, 0.028022943, -0.0048890295, -0.02087984, -0.028922075, 0.046580024, -0.011426467, -0.028747244, -0.0070307115, 0.032268845, -0.0109456815, 0.027398545, -0.008597949, 0.019193968, 0.0034248182, 0.008729072, -0.03903731, 0.013861616, -0.0005303473, -0.0208174, 0.0074615455, -0.004689222, 0.017108481, 0.013212243, -0.00077737443, -0.011463931, -0.014261231, 0.017445656, 0.010689679, -0.03534088, 0.014685821, 0.009022539, -0.030270772, 0.0052324478, -0.011008121, 0.04897771, -0.012369307, -0.0062283613, 0.012406771, 0.03399218, 0.0142237665, 0.03301812, -0.00051863986, 0.027323619, -0.0057819174, -0.0144360615, -0.018856794, -0.01083329, -0.01311234, 0.0066310973, 0.012681506, -0.030195845, 0.006072262, -0.008523021, 0.007411594, 0.054896995, 0.0031719373, 0.015422609, 0.0018966063, 0.03878755, 0.03831301, 0.0042927302, 0.015247778, 0.032069035, 0.02924676, 0.0033311588, 0.014773236, 0.03716412, 0.007592669, 0.002611541, 0.00786116, 0.021991266, 0.0005603964, 0.016608963, 0.02421412, -0.020442761, -0.02142931, 0.0027130055, -0.020680033, -0.016184373, 0.00077698415, -0.0042459005, -0.01650906, -0.0053635715, -0.02355226, 0.001721775, 0.0200931, 0.049327374, -0.032618508, -0.016671402, 0.012700237, -0.013799177, -0.0031219856, -0.013824153, -0.060042027, -0.005878699, -0.051250517, -0.028272701, 0.023290012, 0.010695923, 0.0027254934, 0.028922075, -0.005407279, -0.048678, -0.012269404, -0.007417838, -0.015909638, 0.04031108, 0.013524442, 0.023802018, -0.019693485, -0.027448498, -0.008903903, -0.00038029687, -0.0059317728, -0.023389917, 0.030095942, -0.009522056, 0.027098835, -0.017258337, 0.026449462, -0.018469667, -0.005032641, 0.021079646, 0.012693994], "content_hash": "eb3907a1b6c317fe7a832aec8efad2a2"}]
\ No newline at end of file