Generating Handwriting via Decoupled Style Descriptors

Atsunobu Kotani, Stefanie Tellex, James Tompkin
European Conference on Computer Vision (ECCV) 2020

We synthesize handwriting (bottom) in a target style (top) via learned spaces of style and content,
and can exploit any available reference samples (middle) to improve output quality.

Abstract

Representing a space of handwriting stroke styles includes the challenge of representing both the style of each character and the overall style of the human writer. Existing VRNN approaches to representing handwriting often do not distinguish between these different style components, which can reduce model capability. Instead, we introduce the Decoupled Style Descriptor (DSD) model for handwriting, which factors both character- and writer-level styles and allows our model to represent an overall greater space of styles. This approach also increases flexibility: given a few examples, we can generate handwriting in new writer styles, and also now generate handwriting of new characters across writer styles. In experiments, our generated results were preferred over a state of the art baseline method 88% of the time, and in a writer identification task on 20 held-out writers, our DSDs achieved 89.38% accuracy from a single sample word. Overall, DSDs allows us to improve both the quality and flexibility over existing handwriting stroke generation approaches.

Presentation Video

Presentation Video
(MP4 78.1MB)

Files

Paper arXiv | PDF 2.8MB (PDF 761KB)	Supplemental PDF 5.6MB MP4 5.4MB (below)	Presentation PDF 4.6MB PowerPoint 5.8MB	Code	Dataset ZIP 566.6MB

Bibtex

@inproceedings{kotani2020dsd,
    author = {Kotani, Atsunobu and Tellex, Stefanie and Tompkin, James},
    title = {Generating Handwriting via Decoupled Style Descriptors},
    booktitle = {European Conference on Computer Vision (ECCV)},
    month = {August},
    year = {2020},
}
		

Supplemental Method Explanation Video

(MP4 5.4MB)

Acknowledgements

This work was supported by the Sloan Foundation and the National Science Foundation under award number IIS-1652561. We thank Kwang In Kim for fruitful discussions and for being our matrix authority. We thank Naveen Srinivasan and Purvi Goel for the ECCV deadline snack delivery service. Finally, we thank all anonymous writers who contributed to our dataset.