구름사이

Character: 점(dot)

Text         : 점(Character)의 연속(Sequence) ->  선(Line) 

Image     : 면(surface)

Video      : 면(image)의 연속(Sequence) -> 면체(cube)

LLM( Large Language Model)

• 답변 생성 
• text-to-text, text-to-image

Text-To-Image

• Stable Diffusion (기법 )
  

  

  https://levelup.gitconnected.com/building-stable-diffusion-from-scratch-using-python-f3ebc8c42da3

Text-To-Vedio

• https://levelup.gitconnected.com/building-an-ai-text-to-video-model-from-scratch-using-python-35b4eb4002de 

1. GPU 연산을 줄이고자 한다. 

      - Matix-Multiplication Free LLMs 

       ( https://levelup.gitconnected.com/superfast-matrix-multiplication-free-llms-are-finally-here-cac5b78a4fa7 )

머신 러닝 알고리즘이 비지니스에 활용되는 사례의 검토를 통해,  좀 더 거시적인 측면에서 머신러닝을 이해해 보자.

최신의 신경망( Neural Network )이외의  머신러닝 알고리즘도 다양한 분야에 활용될 수 있다.

머신러닝에 알고리즘 자체에 대한 것은 논하지 않는다.( 전공자라면, 이름과 특성은 익히 또는 종종 들었을 것이다.)

1. Linear Regression

    -  Sales Forecasting: Predicting future sales based on historical data. 

    -   Risk Management: Estimating financial risk by analyzing market trends.

    -  Pricing Strategies: Determining optimal pricing based on various factors such as demand and competition.  

2. Logistic Regression

    - Prediction of Churn: Customers likely to leave a service ( leave or stay )

    - Fraud Detection: Fraudulent vs. Legitimate transactions.

    - Effectiveness of Marketing Campaign: Likelihood of customer response to the campaign

3. Decision Tree

     - Customer Segmentation: Division of customers into several different groups based on their behavior.

     - C redit Scoring: Deciding the creditworthiness of loan applications.

     - Supply Chain Optimization: Making decisions on inventory levels and logistics.

4. Random Forest

      - Product Recommendation—Prediction of products a customer is likely to buy

      - Stock Market Prediction—Prediction of stock price at a given time in the future, given the history

      - Quality Control—Detection of defects in the process of manufacturing

5. Supoort Vector Machines(SVM)

      - Text Classification: Categorizing emails as spam or not spam.

      - Image Recognition: Identifying objects in images.

      - Customer Sentiment Analysis: Classifying customer reviews as positive or negative.

6. K-Nearest Neighbors(KNN)

     - Customer Behavior Analysis: Grouping similar customers for targeted marketing.

     - Recommendation Systems: Suggesting products based on similar user preferences.

     - Anomaly Detection: Identifying outliers in financial transactions.

7. K-Means Clustering

    - M arket Segmentation: Grouping customers based on purchasing behavior.

    - Image Compression: Reducing the number of colors in an image.

    - Document Clustering: Organizing documents into topic-based clusters.

8. Pricipal Component Analysis(PCA)

    - Data Visualization: Simplifying complex datasets for visualization.

    - Feature Extraction: Identifying key features for model building.

    - Noise Reduction: Removing noise from data for better model performance.

9. Neural Networks

     - Image Recognition: Identifying objects and faces in images.

     - Natural Language Processing(NLP): Understanding and generating human language.

     - Fraud Detection: Identifying fraudulent activities in financial transactions.

10. Ensemble Learning  ( such as Bagging, Boosting, Stacking )

     - Fraud Detection: Combining different models to detect fraudulent transactions more accurately.

     - Customer Segmentation: Improving segmentation accuracy by combining various algorithms.

     - Credit Scoring: Enhancing the precision of credit risk assessments.

원본 자료는
https://www.linkedin.com/pulse/essential-machine-learning-algorithms-business-analytics-dictc/

에서 확인할 수 있습니다.

★ 코어 덤프

☆ 코어 덤프 활성화 조회

ulimit -c

☆ 코어 덤프 활성화 조치
 vi /etc/security/limits.conf

 # 모든 사용자에 대해 core file size를 무제한으로 설정
 * soft core unlimited
 * hard core unlimited

저장, 로그아웃, 로그인, 재확인(조회)

☆ 임의로 덤핑
vi InfiniteLoop.java

public class InfiniteLoop {
    public static void main(String[] args) {
        while (true) {
            System.out.println("무한 루프 도는 중...");
            try {
                Thread.sleep(1000);  // 1초 대기 (Optional)
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

kill -11 <pid>

☆ 코어 파일의 경로 및 파일명 조회

cat /proc/sys/kernel/core_pattern

☆ 코어 파일의 경로 및 파일명 설정(임시)

echo "/tmp/core.%e.%p.%h.%t" | sudo tee /proc/sys/kernel/core_pattern

☆ 코어 파일의 경로 및 파일명 설정(영구)

vi /etc/sysctl.conf

kernel.core_pattern = /tmp/core.%e.%p.%h.%t

sudo sysctl -p


★ 파일 디스크립터
☆ 파일 디스크립터 누수 진단 ( 파일은 삭제되었지만 프로세스가 여전히 열고 있을 때, 파일/네트워크소켓/파이트 등)

lsof | grep deleted


☆ 특정 사용자에 의해 열린 파일 나열

lsof -u <username>


☆ 특정 사용자에 의해 열린 파일 개수 파악

#!/bin/bash

USERNAME=$1
USER_UID=$(id -u $USERNAME)
FILE_COUNT=0

for PID in $(ls /proc | grep -E '^[0-9]+$'); do
    PROC_UID=$(stat -c '%u' /proc/$PID 2>/dev/null)
    if [ "$PROC_UID" -eq "$USER_UID" ]; then
        COUNT=$(ls /proc/$PID/fd 2>/dev/null | wc -l)
        FILE_COUNT=$((FILE_COUNT + COUNT))
    fi
done
echo "Total open files by user $USERNAME: $FILE_COUNT"



☆ 특정 프로세스의 열린 파일 디스크립터 수 확인

ls /proc/<PID>/fd | wc -l

lsof -p <PID>


☆ 모든 프로세스의 열린 파일 디스크립터 수 확인

#!/bin/bash
for pid in /proc/[0-9]*; do
    echo -n "$pid: "
    ls "$pid/fd" 2>/dev/null | wc -l
done


lsof | awk '{print $2}' | sort | uniq -c | sort -nr


☆ 파일 디스크립터 한도 조회

ulimit -n 

☆ 파일 디스크립터 한도 조정 

vi /etc/security/limits.conf

* soft nofile 65536
* hard nofile 100000

저장, 로그아웃, 로그인, 재확인 


★ 힙 메모리

☆ 힙메모리 요약 출력

jmap -heap <PID>


☆ JVM 프로세스 ID 확인

jps -l

☆ JVM의 가비지컬렉션(GC)통계를 모니터링

jstat -gc <PID>


☆ GC 로그 활성화

-Xlog:gc*:file=gc.log:time,uptime