Fix concurrency issue in handling references