GPT2 inference with Ray on 2 cores

gpt2_with_ray.py

ray.init(num_cpus=2, ignore_reinit_error=True)

# Create remote function
@ray.remote
def continue_text(beginning):
    text = generator(beginning,  max_length=30, num_return_sequences=1)
    return text[0]['generated_text']

start = time.time()
# Send continue_text() to multiple cores and gather the results 
results = ray.get([continue_text.remote(beginning) for beginning in sentence_beginnings])
print("duration =", time.time() - start)
print(results)

# Output:
# duration = 9.799829721450806
# In Estonia, our main source of energy is from Russian nuclear power plants. But for our new power stations ..
# The main task of the technology scout is finding ways to spot which parts of the code can be used... 
# Renewable energy is important because it can provide an energy source for a wide range of applications...
# Electric Vehicles are changing the lives of people. They're becoming more and more likely to be used as vehicles...