Back to Data Consulting

Minted

Minted is an online marketplace which crowdsources creative content from independent artists and designers to create stationary, wall art, and decor. They host global monthly art and design challenges that are open to and voted on by the public. This enables them to uncover artistic talent and boost their visibility by producing and selling the winning designs. Minted performs analytics on the crowd ratings of each submission in order to promote artist exposure in an industry suited to their style.

Problem

Minted leverages technology to apply the winning artist designs onto products. One of their most substantial sales is greeting cards, designed by the top creators. However, consumers needed to preview their photographs within the card on the website, which required Minted to tag how many pictures should be uploaded by the user. This was a problem because manual tagging consumed many man-hours for the thousands of greeting card designs that are available.

Methodology

Automating photo tagging is a two part process. A computer vision model first needs to identify which parts of an image are photographs, and then use this information to count the number of photos. We developed image recognition algorithms to tackle the first, more complex part of this problem. To generate the correctly pre-processed training data for this model, we explored data segmentation and augmentation techniques, which construct a segmented image to indicate if part of the postcard is photo or template. Then, we designed and tuned a deep learning auto-encoder, which consists of an encoder and decoder to process the image. This model first reduces the photo matrix into a vector to extract features from the compressed representation, and then rebuilds the image using these segmentations.

Results

We delivered an auto-encoder that was trained on images generated from data augmentation. We increased its accuracy by combining our convolutional neural network with various algorithms like Canny and YOLO. Our final model can locate the photos from an image template or user uploaded greeting card. We recommend building a convolutional neural network to count the photos provided from our image segmentation model to finalize the automated pipeline.