I wonder if something like the “secret language” (or whatever the more correct term is) also emerges from traditional (pure-text) large language models. A good thread from @Plinz explaining the “secret language” phenomenon that is making the rounds, that agrees with my own exps.