Pattern Tracking with Machine Learning

In the world of technical analysis, recognizing chart patterns is a critical skill for investors. Instead of manually scanning charts, I aimed to automate this process by training a YOLOv8 deep learning model to detect key formations such as Head and Shoulders, Double Top, Inverted Double Top, Ascending Triangle, and Descending Triangle.

In this article, I will provide a detailed explanation of how I collected and labeled data, trained the YOLOv8 model, and evaluated the results.

Data Preparation

Creating a high-quality dataset is essential for training a deep learning model. Since there was no publicly available dataset that met my needs, I had to create my own.

Step 1: Generating Candlestick Charts

I wrote a Python script that performs the following tasks:

Random Selections:

Randomly selects one of the USDT trading pairs from Binance.

Randomly selects a specific time interval (e.g., 1D, 4H, 1H, 15M, 5M).

Randomly selects a number of candlesticks between 10 and 70.

Fetching Data:

Retrieves klines data from Binance API based on the selected trading pair, time interval, and number of candlesticks.

Generating and Saving Charts:

Processes the retrieved data, plots the candlestick chart, and saves it as an image.

chart

You can access the code here:

https://github.com/esprator07/patern_recognition_with_yolov8/blob/main/chart_maker.py

Step 2: Labeling the Patterns

After generating a sufficient number of candlestick charts, I needed to label the ones that contained technical patterns. I performed the labeling by recording the exact location of the pattern in the image as a TXT file.

Example Label Format:

Ex: 4 0.3134765625 0.275390625 0.169921875 0.2109375

• The first number (4) represents the pattern class.

• The other numbers define the bounding box coordinates in the image.

I developed my own labeling software, but for those looking for an easier approach, Roboflow can be a great alternative.

Dataset Summary

• Total Collected Images: 1000

• Number of Pattern Classes: 5

Pattern Classes:

• Head and Shoulders

• Double Top

• Inverted Double Top

• Ascending Triangle

• Descending Triangle

Training the YOLOv8 Model

After preparing the dataset, I uploaded it to Google Drive and trained the model using Google Colab.

Step 1: Connecting Google Drive and Loading Data

To access the dataset stored in Google Drive from Google Colab, I used the following command:

from google.colab import drive
drive.mount('/content/drive')

Then, I copied the YAML file to the working directory:

!cp /content/drive/MyDrive/dataset.yaml /content/

The YAML file stored in Google Drive was as follows:

train: /content/datasets/dataset1/images/train
val: /content/datasets/dataset1/images/val
nc: 5
names: ["ascending_triangle", "descending_triangle", "double_top", "head_and_shoulders", "inverted_double_top"]

Then, I uploaded the dataset’s ZIP file and extracted it:

!cp /content/drive/MyDrive/datasets.zip /content/
!unzip /content/datasets.zip -d /content/

Step 2: Installing Required Libraries

To run the YOLOv8 model, I installed the following libraries:

!pip install ultralytics
!pip install torch torchvision torchaudio
!pip install opencv-python-headless
!pip install numpy matplotlib

Step 3: Model Training Parameters

I trained the YOLOv8s model using the following settings:im:

from ultralytics import YOLO
model = YOLO("yolov8s.pt")  
model.train(
    data="dataset.yaml",

    device=0,  # Use GPU (Avoid CPU for faster training)
    epochs=300,  # Train for more epochs
    batch=8,  # Smaller batch size for small datasets
    imgsz=512,  # Larger input image size
    optimizer="SGD",  # More stable learning with SGD optimizer
    patience=70,  # 🛑 Early stopping to prevent unnecessary training
    lr0=0.004,  # Lower initial learning rate
    dropout=0.2,  # Prevent overfitting
)

Step 4: Training Process

Used GPU: Google Colab T4 GPU

Training Duration: Approximately 2 hours

Results:

Precision: 76%

mAP50 (Mean Average Precision @50 IoU): 72%

⚠ I did not use any augmentation techniques, as rotating or changing the colors of candlestick charts would not make sense.

You can find more details about the model below:

You can access the trained model from the following link:

https://github.com/esprator07/patern_recognition_with_yolov8/blob/main/trainsmall.pt

⚠ Since the model was trained using my own generated charts and no augmentation was applied, it may perform inefficiently on some chart images. ⚠

Using the Trained Model & Results

After completing the training, I exported the model in .pt format:

Then, I tested the model on a dataset containing 4,102 chart images. The model identified formations and saved them in a separate folder for further analysis.

Results:

Total Examined Files: 4,102

Files Containing Formations: 570

Pattern Types & Distribution

Pattern Type	Count
Ascending Triangle	132
Descending Triangle	127
Inverted Double Top	184
Double Top	59
Head and Shoulders	68

The formations detected by the model are shown in the example images below:

Double Top (Confidence %82)

Inverted Double Top (Confidence %92)

Head and Shoulders (Confidence %97)

Ascending Triangle (Confidence %80)

Descending Triangle (Confidence %82)

False Detections & Model Behavior

🔹 While the model performed well overall, it sometimes made incorrect detections.🔹 A larger dataset and improved labeling can help reduce errors.

Conclusion

Training a YOLOv8 model for chart pattern detection was an exciting project. With 76% precision and 72% mAP50, the model performed quite well, but there is still room for improvement.

During the testing phase, 4,102 different charts were analyzed, and 570 of them contained detected patterns. This indicates that the model can identify formations with a reasonable accuracy rate. The most frequently detected formation was Inverted Double Top (184 instances), while the least detected formation was Double Top (59 instances). This distribution suggests that the model recognizes some patterns better than others, highlighting the need for further data augmentation to balance the dataset.

As the next step, I plan to develop a user-friendly interface (GUI) to make the model more practical. With this interface, users will be able to perform pattern detection more easily, analyze charts in real-time, and review results more efficiently. Once the interface is ready, I will update this document with a new article explaining its implementation.

Additionally, in the future, I plan to retrain the model with a larger dataset, reduce data imbalance, and compare it with other AI algorithms. These improvements will make the model more reliable and accurate in detecting chart patterns.