The charity says the gastropub near the former home of the Bard's wife is "detrimental" to the area.