Posts by Category

Tech 41
DailyLife 17

Tech

Serving a simple chat model into an API

It has been a long time since I posted a tech post!

Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances

Let me introduce an interesting paper I have recently read regarding dialogue modeling and evaluation.

DialogueSentenceBERT: SentenceBERT for More Representative Utterance Embedding via Pre-training on Dialogue Corpus (2)

Previous post: https://songstudio.info/tech/tech-38

DialogueSentenceBERT: SentenceBERT for More Representative Utterance Embedding via Pre-training on Dialogue Corpus (1)

In the 2nd quarter of this year, I conducted research on developing a contextualized sentence embedding model, which is more optimized on an utterance embedding for dialogue tasks.

Averaging methods for F1 score calculation in multi-label classification

There are various evaluation metrics to test the model we trained when conducting machine learning projects.

DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

It’s been a long time since I’ve posted a review on an NLP paper.

Multi-turn chatbot project (3): GPT-2 chatbot with multi-turn generation settings

(This post was modified in December 2nd after the re-implementation of the project to prevent you from confused by the difference between the repository and the contents of the post.)

Multi-turn chatbot project (2): Transformer chatbot with the ReCoSa structure

Following the introductions that I posted last time, today let’s talk about the transformer model using the ReCoSa(the Relevant Contexts with Self-attention) structure, which is the first model for the multi-turn chatbot.

Multi-turn chatbot project (1): Introductions

Starting with this, I’m gonna post about the personal project developing the “Multi-turn chatbot”.

Computing Machinery and Intelligence

Few of those majoring in computer science would not know “Alan Turing”.

Bias-Variance trade-off

“Bias-Variance trade-off” is one of the fundamental concepts in Machine Learning studies, which means that there is a trade-off relation between two errors(or losses), bias and variance when evaluating the generalization capacity of ML algorithms.

Revisiting the transformer NMT project

It’s been a while.

The Curious Case of Neural Text Degeneration

This post is the review of the publication, Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019). The curious case of neural text degeneration.

Beam Search

Beam Search is a tree search algorithm based on “Best First Search” method used in various NLP tasks frequently.

Monty Hall problem

“Monty Hall problem” is a very famous mathematical problem related to conditional probabilities.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

BERT stands for “Bidirectional Encoder Representations from Transformers” which is one of the most notable NLP models these days.

Neural Machine Translation with Transformer in Pytorch

We reviewed the famous Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008) last time.

Attention Is All You Need

This post is about the famous Transformer, which has advanced the progress of NLP research.

Union-Find Algorithm

https://songstudio.info/tech/tech-17/

Attention mechanism in simple RNN based model

Since the basic idea seeing the overall input contexts as references is same, it is obvious that we can use attention to basic RNN based models.

Attention mechanism in seq2seq model

Attention mechasnism is one of the most important concepts in NLP field.

Sequence-to-Sequence model

Sequence-to-Sequence(seq2seq) model is a Deep Learning model usually used for machine translation, text summarization etc.

2019-2 YBIGTA CONFERENCE

YBIGTA is the Big Data Academy in the college of Engineering, Yonsei University.

AICON 2019

It has been quite a long time, but I finally managed to post about AICON 2019, which is a global AI conference held at Yangjae R&D Innovation Hub, in Dec 17th, 2019.

Minimum Spanning Tree

Spanning Tree is a tree that has all vertices from an original graph and has a minimum number of edges.

Mean Square Error & Cross Entrophy Loss

The most commonly used loss functions in Machine Learning/Deep Learning are Mean Square Error and Cross Entropy loss.

Traveling Salesman Problem

Traveling Salesman Problem(TSP) is one of the most famous problems in algorithm and the basic example is below.

Finding the Diameter of a Tree

https://www.acmicpc.net/problem/1967

Operator Overloading

Simply saying, this is about re-defining basic operators for primitive types and it can be used for operating between objects from classes or structs.

Priority Queue

I found the data structure called “Priority Queue” in <queue> libaray.

Segment Tree

https://www.acmicpc.net/problem/17022

Default values in a map

https://songstudio.info/tech/tech-4/

Making Pytorch custom dataset

When doing simple practices, we can download datasets provided by the framework itself, process them into loaders and put them into our models.

Thoughts about time and memory

https://www.acmicpc.net/problem/14217

The difference between DFS and BFS

Today I solved a problem which should be approached with BFS.

Runtime error

During these several days, I have suffered from runtime errors while solving Baekjoon Online Judge problems.

Using partial sums

https://www.acmicpc.net/problem/14465

Updating the value in map

I have usually used map.insert() to put a value in the map and known that this would update automatically if there is already the existing key value.

Finding cycles in graph application

Until now, I have thought that I have become accustomed to solving problems clearly with graph searching, for instance, DFS, BFS. But I realized that a problem that doesn’t seem to be solved with a graph can be processed with a graph structure.

Binary search

Binary search is a powerful method to find the desired value in a sorted sequence, but I have not been used to decide whether the given case should be solved with Binary search.

next_permutation

I noticed that there is a function to get permutations of a given sequence.

Tech

DailyLife