Thoughts on Objectives of Sparse and Hierarchical Masked Image Model
By: Asahi Miyazaki, Tsuyoshi Okita
Potential Business Impact:
Teaches computers to understand pictures better.
Masked image modeling is one of the most poplular objectives of training. Recently, the SparK model has been proposed with superior performance among self-supervised learning models. This paper proposes a new mask pattern for this SparK model, proposing it as the Mesh Mask-ed SparK model. We report the effect of the mask pattern used for image masking in pre-training on performance.
Similar Papers
Evolved Hierarchical Masking for Self-Supervised Learning
CV and Pattern Recognition
Teaches computers to see details better.
The Efficiency of Pre-training with Objective Masking in Pseudo Labeling for Semi-Supervised Text Classification
Computation and Language
Teaches computers to learn from less labeled text.
Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understanding
CV and Pattern Recognition
Teaches computers to understand 3D worlds without labels.