Hangfire Dashboard

TimesheetSyncTask.Execute

// Job ID: #754644
using Application.Sync;

var timesheetSyncTask = Activate<TimesheetSyncTask>();
await timesheetSyncTask.Execute();
CurrentCulture
"en-US"
CurrentUICulture
"en-US"
RecurringJobId
"TimesheetSyncTask"
RetryCount
10

State

9/3/2025 3:48:13 PM (+56.384s) Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+93ms Processing

Server:
ec2amaz-sbdjgq5.default:2528
Worker:
7c5ec570

+1h 50m Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+7ms Scheduled

Retry attempt 10 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 3:47:15 PM

+56.707s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+110ms Processing

Server:
ec2amaz-sbdjgq5.default:5116
Worker:
4c1573d3

+1h 10m Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+6ms Scheduled

Retry attempt 9 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 1:56:02 PM

+55.920s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+46s Processing

Server:
ec2amaz-sbdjgq5.default:1240
Worker:
015e464e

+90ms Processing

Server:
ec2amaz-sbdjgq5.default:2656
Worker:
85374d24

+44m 10.244s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+6ms Scheduled

Retry attempt 8 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 12:43:50 PM

+54.254s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+73ms Processing

Server:
ec2amaz-sbdjgq5.default:2608
Worker:
2a5382ad

+24m 53.113s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+10ms Scheduled

Retry attempt 7 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:58:34 AM

+52.114s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+116ms Processing

Server:
ec2amaz-sbdjgq5.default:556
Worker:
77cc8cf2

+13m 35.654s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+6ms Scheduled

Retry attempt 6 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:32:55 AM

+51.084s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+83ms Processing

Server:
ec2amaz-sbdjgq5.default:3176
Worker:
bd68f48c

+5m 58.513s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+7ms Scheduled

Retry attempt 5 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:18:33 AM

+52.063s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+9.617s Processing

Server:
ec2amaz-sbdjgq5.default:3176
Worker:
bd68f48c

+2m 20.430s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+7ms Scheduled

Retry attempt 4 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:11:31 AM

+54.613s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+60ms Processing

Server:
ec2amaz-sbdjgq5.default:3176
Worker:
bd68f48c

+1m 59.327s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+3ms Scheduled

Retry attempt 3 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:08:19 AM

+52.160s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+8.627s Processing

Server:
ec2amaz-sbdjgq5.default:3176
Worker:
bd68f48c

+51.436s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+4ms Scheduled

Retry attempt 2 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:05:11 AM

+53.593s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+47ms Processing

Server:
ec2amaz-sbdjgq5.default:3176
Worker:
bd68f48c

+40.120s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+3ms Scheduled

Retry attempt 1 of 10: Access Denied: The Personal Access Token used has…

Enqueue at:
9/3/2025 11:03:32 AM

+1m 19.833s Failed

An exception occurred during performance of the job.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException

Access Denied: The Personal Access Token used has expired.

Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: Access Denied: The Personal Access Token used has expired.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationsAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<EnsureResourceLocationsPopulated>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<GetResourceLocationAsync>d__63.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<CreateRequestMessageAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<Query>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TeamServices.WorkItemMaterializer.<<Query>b__5_0>d.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemMaterializer.cs:line 37
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at TeamServices.WorkItemRepository.<FindByReference>d__6`1.MoveNext() in C:\TeamServices\Agent\_work\555\s\TeamServices\WorkItemRepository.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Subjects.AsyncSubject`1.GetResult() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Subjects\AsyncSubject.cs:line 437
   at Application.Sync.TimesheetSyncTask.<Execute>d__6.MoveNext() in C:\TeamServices\Agent\_work\555\s\Application\Sync\TimesheetSyncTask.cs:line 56

+12h 1m Processing

Server:
ec2amaz-sbdjgq5.priority:3176
Worker:
6d22d085

+14.496s Processing

Server:
ec2amaz-sbdjgq5.priority:1984
Worker:
4bc17cd5

+37ms Enqueued

Triggered by recurring job scheduler

Queue:
priority

9/2/2025 11:00:01 PM Created