NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems...

次の動画（詳細編）：https://youtu.be/kF3v3_hsWUQ文章に沿った画像を高品質かつ多様に生成することが可能なDALL-E 2の概要について紹介しています。DALL-E 2のプロジェクトページへのリンク：https://openai.com/dall-e-2/--ソニーが提供するオープンソース...

What components are needed for building learning algorithms that leverage the structure and properties of graphs?

In an effort to reduce the computational load of Transformers, research on linear attention has gained significant momentum. However, the improvement strategies for attention mechanisms typically...

Transformer is a powerful architecture that achieves superior performance on various sequence learning tasks, including neural machine translation, language understanding, and sequence prediction....

I would not have a data science career without Kaggle. So if you are looking for a blog post bashing Kaggle, this is not the place. Competing on Kaggle is worth it! It will teach you a lot about...

A typical PyTorch training program on 8 GPUs with 4 dataloader workers per GPU would create at least processes. A naive use of PyTorch dataset and dataloader can easily replicate your dataset's

Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the...

近年、単語をベクトル表現する埋め込み技術が広く用いられていますが、単語を「点」でしか表現できず、概念の広がりや階層関係を表現できないという問題がありました。そこで、データを「箱」などの領域によって表現する埋め込み手法が研究されています。本記事では、データを箱として表現し、ベクトルよりも強力な埋め込みを学習する「Box Embedding」について基礎から丁寧に解説します。

An efficient video loader for deep learning with smart shuffling that's super easy to digest - dmlc/decord

「学習率をバッチサイズに比例して増やす」（ルール1） という慣例があるが、これの根拠について説明する。 ��拠 まず、このアイデアの出所は論文[1]だと思われる。 この論文では「学習率をバッチサイズに比例して増やす」というルールを適用すると幅広いバッチサイズで収束後の性能が良くなることを経験的に示した。 論文ではこの法則…

【AI論文解説】はディープラーニング・機械学習に関する論文を紹介する動画シリーズです。（プレイリスト： https://www.youtube.com/playlist?list=PLbtqZvaoOVPCqfmnrBfo9Xv5mtDr0LjQZ ）今回は、ロボット版GPTともいえるRobotics Trans...

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN). - Blealtan/efficient-kan

Despite their impressive capabilities, large language models (LLMs) are prone to hallucinations, i.e., generating content that deviates from facts seen during pretraining. We propose a simple...

Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classification, speech...

Underlying the use of statistical approaches for a wide range of applications is the assumption that the probabilities obtained from a statistical model are representative of the "true"...

Text generation is of great importance to many natural language processing applications. However, maximization-based decoding methods (e.g. beam search) of neural language models often lead to...

Improve high precision GNSS positioning and navigation accuracy on smartphones

TL;DR半教師あり学習の1つの手法である、疑似ラベルをCIFAR-10で試したサンプル数が少ない場合は、疑似ラベルを使うことでテスト精度を引き上げることができたただし、転移学習と比べると若干…

A simple extension for Jupyter Notebook and Jupyter Lab to beautify Python code automatically using black. - dnanhkhoa/nb_black

The current trend of scaling language models involves increasing both parameter count and training dataset size. Extrapolating this trend suggests that training dataset size may soon be limited by...

Open-source version control system for Data Science and Machine Learning projects. Git-like experience to organize your data, models, and experiments.

It’s probably a 0 division somewhere. Have you tried using a much larger eps (say 1e-4)? The default 1e-8 is rounded to 0 in half precision.

A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to facilitate metric computation in distributed training and tools...

The scaling of Transformers has driven breakthrough capabilities for language models. At present, the largest large language models (LLMs) contain upwards of 100B parameters. Vision Transformers...

Multiple Additive Regression Trees (MART), an ensemble model of boosted regression trees, is known to deliver high prediction accuracy for diverse tasks, and it is widely used in practice....

Pet adoption ML coder apologizes and says desire to be ranked #1 'compromised my judgement'

Summary Right now if we try to train a folder of image data that cannot fit into memory of a single machine, lightgbm is not an available option. We can see some distributed solutions for lightgbm ...

Training object class detectors typically requires a large set of images in which objects are annotated by bounding-boxes. However, manually drawing bounding-boxes is very time consuming. We...

Transformer has shown great successes in natural language processing, computer vision, and audio processing. As one of its core components, the softmax attention helps to capture long-range...

High-quality text embedding is pivotal in improving semantic textual similarity (STS) tasks, which are crucial components in Large Language Model (LLM) applications. However, a common challenge...

[This image will be available on Kaggle Notebooks in the next few days] gcr.io/kaggle-private-byod/python:v128 == Comparing pip list --format=freeze == astroid==2.14.1 | astroid==2.14.2 ...

11月から参加していたKaggleコンペOTTO – Multi-Objective Recommender System（通称Otto）の期間中にあった出来事について振り返っておく。 全体の感想 CVのリークについて データ設計について 共通の設計 学習スキーム（candidate generator; 1段目） 学…

We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance....

Use LLMs to answer difficult science questions

The merlin dataloader lets you rapidly load tabular data for training deep leaning models with TensorFlow, PyTorch or JAX - NVIDIA-Merlin/dataloader