Skip to content

Instantly share code, notes, and snippets.

@Birch-san
Last active July 10, 2023 17:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Birch-san/18431b4e549a16c1a4054e61370ebfaf to your computer and use it in GitHub Desktop.
Save Birch-san/18431b4e549a16c1a4054e61370ebfaf to your computer and use it in GitHub Desktop.
Correctness-testing bitsandbytes `0.40.0`

correctness-testing 0.40.0

Here we've ramped up the bnb_4bit_compute_dtype to float32, in the hopes of making the model stay on-topic.
Since we were concerned by the responses measured with bnb_4bit_compute_dtype=bfloat16

llama 7b

I was under the effect of a counterspell, so none of the superpower-wielding monsters could see me anyway. My eyes had begun to change as a result of my battle with Melvin. The transformation was complete. I was in the true look of my chosen form. As you can see, a true-blue beauty. There was only one of me, though, so I would have to make sure that this was the end. I went to catch the culprit. He was in the same clothes he was wearing when he committed the first murder. I did not recognize the man from that time, nor did he from me, but his face was twisted with an evil grin. He had the same shaved head. However, his hair seemed to change color. It was dark brown when I met him, but it turned to blonde. He also had on a black suit, a pair of sunglasses, and an earring in his left ear. As I moved in for another strike, he made his move, too. He used an object, though, which I was unable to identify. The blade of light shot me, the force of such was enough to send me flying. I rolled

ctx length: 25
tokens out: 256
duration: 9.90 secs
speed: 25.85 tokens/sec

llama 13b

I was under the effect of a spell at the moment. No one would recognise my face, which had been changed to that of an eighteen-year-old girl at the time when it hadn't really been a transformation. I was walking toward the room of my new roommate. Before doing so, I looked back at the garden. "S-so, I'm gonna go sleep, too. My shift is over next morning." The security guard standing opposite the pool was saying that to me, a robed woman. That bodyguard of Mistress Mio's was standing still and calm after I had passed by. "Yeah, that's true. Tomorrow, we're supposed to go sightseeing, right?" I asked that to the woman who was not even glancing at me. My eyes were shining under my eyeglasses, but there were no expressions of feelings toward her on my face. As a result, she nodded slightly and said "Yeah…" in an unsure fashion, which made up her answer. She had a strong sense of security, the girl Mio. I saw her at a glance. That woman

ctx length: 25
tokens out: 256
duration: 11.90 secs
speed: 21.51 tokens/sec

llama 30b

I was under the effect of a spell at the moment. No one would suspect anything different of me if I was to leave my home and go out chasing the criminal. Then the policemen would be able to do their jobs. I would soon be on the case. I was to do so in order to protect people against magic-using criminals. My mission began here, with that thought in mind. What I did first was to gather all the information I could. There were two kinds. One was the knowledge of the area, and the other was the information on the culprit. I knew the area well enough, thanks to being from here, but the culprit was a different story. I was unsure of their whereabouts or if there were any allies among them. I could not ignore the fact that their actions had been planned to be carried out in the shadows of night with the full protection of darkness. Their aim was to cause the most damage in the shortest amount of time possible before retreating. On top of all this, it was a magic-using criminal, which meant that they had the potential to be very strong. One thing that they did not possess, however, was a large reserve of magic

ctx length: 25
tokens out: 256
duration: 17.68 secs
speed: 14.48 tokens/sec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment