Amazon.com Inc. is pushing deeper into the grocery business with plans to introduce convenience stores as well as curbside pickup locations, say people familiar with the matter.

The Seattle company aims to build small brick-and-mortar stores that would sell produce, milk, meats and other perishable items that customers can take home, these people say. Primarily using their mobile phones or, possibly, touch screens around the store,…