ASAC 60일차_딥러닝 13일차

2024.02.27

 

NLP : Transformer

• RNN ⇒ LSTM ⇒ Transformer
• 2017년도에 구글에서 tranformer를 개발하면서 제일 기반이 되는 언어 모델의 시작이 되었다.
• 그 다음으로 BERT가 유행했고, 2020년 GPT3의 등장으로 이어지고 있다. 
• 이런 모델들을 LLM으로 통칭하고 있다.

 

Transformer 실습

자연어 처리 교재의 챕터별 내용 다운

!git clone https://github.com/rickiepark/nlp-with-transformers.git
%cd nlp-with-transformers
from install import *
install_requirements(chapter=1)

 

0. 예제 텍스트 준비

예제 텍스트: 아마존 항의 메일

text = """Dear Amazon, last week I ordered an Optimus Prime action figure \
from your online store in Germany. Unfortunately, when I opened the package, \
I discovered to my horror that I had been sent an action figure of Megatron \
instead! As a lifelong enemy of the Decepticons, I hope you can understand my \
dilemma. To resolve the issue, I demand an exchange of Megatron for the \
Optimus Prime figure I ordered. Enclosed are copies of my records concerning \
this purchase. I expect to hear from you soon. Sincerely, Bumblebee."""

 

1. 텍스트 분류 모델

Hugging Face에 가입 후, 토큰을 발급 받는다. 

토큰을 활용한 API access 과정을 거친다.

from transformers import pipeline
access_token= # "발급받은 토큰 입력"
classifier = pipeline("text-classification",token=access_token)

outputs = classifier(text)
print(outputs)
pd.DataFrame(outputs)

해당 텍스트를 분류 모델에 넣었더니 텍스트의 종류와 점수를 출력한다. 

텍스트 상에 있는 객체들에 대한 정보 추출

# Edited : token 추가
ner_tagger = pipeline("ner", aggregation_strategy="simple",token=access_token)
outputs = ner_tagger(text)
print(text)
print("*"*50)
print(outputs)
print("*"*50)
pd.DataFrame(outputs)

 

---

2. Q & A

ChatGPT 까지는 아니지만 이미 학습된 것을 가져와 사용해본다. 

해당 항의 메일에서 고객이 원하는 것이 무엇인지 질문해본다. 

reader = pipeline("question-answering",token=access_token)

# 질문
question = "What does the customer want?"
outputs = reader(question=question, context=text)
print(outputs)
pd.DataFrame([outputs])

 

question = "Where did he buy the Megatron?"
outputs = reader(question=question, context=text)
print(outputs)
pd.DataFrame([outputs])

단, 성능이 엄청 좋은 모델은 아니기에 조금 애매한 경우도 있다. 

해당 질문의 실제 답은 '독일의 온라인 아마존 매장'이지만, '독일'로 답변하는 모습을 볼 수 있다.

 

---

3. 문서 요약

summarizer = pipeline("summarization",token=access_token)

outputs = summarizer(text, max_length=30, clean_up_tokenization_spaces=True)
print(outputs[0]['summary_text'])


# 결과
Bumblebee ordered an Optimus Prime action figure from your online store in
Germany. Unfortunately, when I opened the package, I discovered to

 

---

4. 번역

영어로 작성된 메일을 독일어로 번역

translator = pipeline("translation_en_to_de",
                      model="Helsinki-NLP/opus-mt-en-de",token=access_token)
outputs = translator(text, clean_up_tokenization_spaces=True, min_length=100)
print(outputs[0]['translation_text'])


# 결과
Sehr geehrter Amazon, letzte Woche habe ich eine Optimus Prime Action Figur aus
Ihrem Online-Shop in Deutschland bestellt. Leider, als ich das Paket öffnete,
entdeckte ich zu meinem Entsetzen, dass ich stattdessen eine Action Figur von
Megatron geschickt worden war! Als lebenslanger Feind der Decepticons, Ich
hoffe, Sie können mein Dilemma verstehen. Um das Problem zu lösen, Ich fordere
einen Austausch von Megatron für die Optimus Prime Figur habe ich bestellt.
Eingeschlossen sind Kopien meiner Aufzeichnungen über diesen Kauf. Ich erwarte,
von Ihnen bald zu hören. Aufrichtig, Bumblebee.

 

+추가 ) 한국어 영작

translator2 = pipeline("translation", model="Helsinki-NLP/opus-mt-ko-en",token=access_token)
translator2("오늘의 날씨가 참 좋습니다")

# 결과
[{'translation_text': "Today's weather is so good."}]

 

---

5. 텍스트 생성

항의 메일에 대한 적당한 답변 생성하기

from transformers import set_seed
set_seed(42) # 동일 결과를 재현하기 위해 지정

generator = pipeline("text-generation",token=access_token)

# 프롬프트 작성
response = "Dear Bumblebee, I am sorry to hear that your order was mixed up."
prompt = text + "\n\nCustomer service response:\n" + response

outputs = generator(prompt, max_length=200)
print(outputs[0]['generated_text'])

# 결과
Dear Amazon, last week I ordered an Optimus Prime action figure from your online
store in Germany. Unfortunately, when I opened the package, I discovered to my
horror that I had been sent an action figure of Megatron instead! As a lifelong
enemy of the Decepticons, I hope you can understand my dilemma. To resolve the
issue, I demand an exchange of Megatron for the Optimus Prime figure I ordered.
Enclosed are copies of my records concerning this purchase. I expect to hear
from you soon. Sincerely, Bumblebee.

Customer service response:
Dear Bumblebee, I am sorry to hear that your order was mixed up. Please inform
me immediately that it was not completed. I will try to locate the items you
ordered in your shop on the day. As soon as I have received that information
(this was only on Wednesday which means that it could take too long to process
the results of the exchange), I will refund the order and refund my entire
purchase price