To address this vulnerability, PFCP completed emergency maintenance on July 23, 2026, during which we updated the Linux kernel on the host operating system of PFCP-managed Kubernetes nodes.

Mitigation

The Linux kernel on the host operating system (Ubuntu 24.04 LTS) of PFCP-managed Kubernetes nodes has been updated to 6.8.0-136.136, which addresses GhostLock (CVE-2026-43499).

Completion Time

July 23, 2026, 12:00 JST

Customer Action Required

No action is required from PFCP users.

For details of the maintenance associated with this update, please refer to the following announcement.

• https://pfcomputing.com/news/20260721-2/

References

• Ubuntu Security Notice: https://ubuntu.com/security/CVE-2026-43499

PFCP is committed to providing timely security updates so that our users can continue to use our services with confidence.

Scheduled Maintenance Date:

Thursday, July 23, 2026

Time:

9:00 – (all day)

Affected Service:

Kubernetes Service (including the Workspace feature)

Affected Sites:

SR1, IK1, YH1, IZ1

• AW1 is not covered by this maintenance. Affected users at this site will be contacted individually.

Impact:

As with cluster upgrades, compute nodes will be rebooted sequentially (up to 25% at a time) rather than all at once.

• Compute nodes will be rebooted to apply the kernel update. Applications (Pods) running on Kubernetes and Workspace execution environments will be temporarily stopped, deleted, and recreated.
• Pods created directly without a higher-level resource such as a Deployment or Job will not be recreated automatically.
• Data stored in persistent storage (PVCs, etc.) is not affected. However, data not in persistent storage — including data inside Pods and Workspace execution environments — will be lost.
• Reserved nodes allocated to users will be temporarily unavailable while they are being rebooted.
  • If you have fewer than 4 reserved nodes, they will be rebooted one at a time. If you have 4 or more, up to 25% (rounded down) will be temporarily unavailable at a time.
  • If you have only 1 reserved node, it will be temporarily unavailable.
• Up to 25% (rounded down) of shared nodes will be temporarily unavailable.

Reason:

To update compute nodes to a Linux kernel version that includes a fix for the discovered vulnerability.

Action requested of users:

• If you use Kubernetes services or the Workspace feature, save any necessary data to persistent storage before maintenance begins.
• If you need to preserve in-progress computation state, complete checkpointing before maintenance starts.
• If you have created Pods directly without a higher-level resource such as a Deployment or Job, those Pods will not be recreated automatically. Please review your configuration in advance.

This maintenance is an emergency maintenance conducted in response to a critical security vulnerability. For our maintenance policy, see the User Guide.

Scheduled Maintenance Date:

Monday, August 3, 2026

Network Equipment Maintenance

Time:

9:00 – 11:00

Affected Service:

Kubernetes Service

Affected Sites:

All sites

Impact:

Network connectivity may be temporarily unstable, or Pod creation may be temporarily unstable.

Reason:

Network component version upgrade.

Storage Equipment Maintenance

Time:

9:00 – 18:00

Affected Service:

Kubernetes Service (Persistent Storage)

Affected Sites:

SR1, YH1

Impact:

Response times may be slightly delayed during the maintenance window.

Reason:

Storage appliance OS update.

For our maintenance policy, see the User Guide.

You can obtain the Markdown version simply by changing the file extension in the URL from .html to .md.

• HTML version: https://docs.pfcomputing.com/en/glossary.html
• Markdown version: https://docs.pfcomputing.com/en/glossary.md

A “View as Markdown” button has also been added to the menu bar of each page, which takes you to the Markdown version when clicked.

This is useful when Markdown is easier to work with than HTML, such as when feeding documentation to an AI agent.

• English version: https://pfcomputing.com/en/news/

You can switch between Japanese and English versions using the language selection menu at the bottom of the page.

• English version: https://docs.pfcomputing.com/en/

You can switch between Japanese and English using the language switcher menu in the upper right corner of the page.

The English version has been created based on machine translation, with the Japanese version as the authoritative source. If there are any discrepancies, please refer to the Japanese version.

For details, see the User Guide.

Scheduled Maintenance Date:

Monday, July 6, 2026

Time:

9:00 – 10:00

Affected Service:

Kubernetes Service

Affected Sites:

IK1

Impact:

Network connectivity may be temporarily unstable, or Pod creation may be temporarily unstable.

Reason:

Network component version upgrade.

For our maintenance policy, see the User Guide.

Scheduled Maintenance Date:

Monday, June 15, 2026

Time:

9:00 – 11:00

Affected Service:

Kubernetes Service

Affected Sites:

All sites

Impact:

Network connectivity for Kubernetes nodes may be temporarily interrupted or unstable.

Reason:

Configuration changes required by Cilium API changes.

For our maintenance policy, see the User Guide.

For details, see the User Guide.

The Kubernetes version has been updated from v1.34.3 to v1.35.3. For details on what changed, see the Kubernetes official release blog entry.

The updated software versions are as follows.

Clusters and Add-ons

Monitoring Service

• Software Versions - Preferred Computing Platform (PFCP) User Guide

PFCP performs regular software version upgrades to improve the stability, performance, and security of Kubernetes and other components. For details, see the PFCP User Guide / Maintenance Policy.

Under certain conditions, HTTP access through PFCP’s Identity-Aware Proxy could have allowed access to Pods in namespaces that the user was not authorized to access.

In response, the ExternalName Service feature used in the affected access path has been disabled, and the related authentication settings have been reviewed. This issue has been resolved.

Mitigation:

• Disabled ExternalName Service.
• Reviewed related authentication settings.

Completion Date:

May 12, 2026

User Impact:

No action is required from users. You can continue using the service as usual.

PFCP remains committed to maintaining and improving security so that users can use the service with confidence.

Scheduled Date:

Monday, June 15, 2026

Planned Work:

Details of the work to be performed will be announced at least two weeks before the maintenance date, in accordance with our regular maintenance policy.

Note:

The regular scheduled maintenance date is the first Monday of each month. This announcement is being made because the date differs from the usual schedule. No further notice will be sent if the date remains unchanged.

For our maintenance policy, see the User Guide.

Mitigation:

Disabled the vulnerable feature.

Completion Time:

May 8, 2026, 12:00 JST

User Impact:

No action is required from users. You can continue using the service as usual.

Vulnerability Details:

No CVE has been assigned, but please refer to the following information.

• https://blog.cloudlinux.com/dirty-frag-mitigation-and-kernel-update

PFCP is committed to providing prompt security updates so that users can use the service with confidence.

Mitigation:

Disabled the vulnerable feature.

Completion Date:

April 30, 2026

User Impact:

No action is required from users. You can continue using the service as usual.

Vulnerability Details:

• CVE-2026-31431: https://nvd.nist.gov/vuln/detail/CVE-2026-31431

PFCP is committed to providing prompt security updates so that users can use the service with confidence.

This incident was resolved at approximately 10:30 on April 28, 2026.

Time of Occurrence:

Approximately 10:22 on April 28, 2026

Time of Resolution:

Approximately 10:30 on April 28, 2026

Affected Services:

• PFCP Portal
• Kubernetes Service
  • Kubernetes cluster (internet access only)
  • WebApp Identity-Aware Proxy (Ingress)
  • API Identity-Aware Proxy (API IAP)
• Monitoring Service
  • Grafana

Affected Sites:

All sites

Impact:

Access or connections to the above services were failing.

Workloads running on site clusters were not directly affected.

Cause:

An operational error during maintenance work caused part of the systems responsible for PFCP’s network relaying to stop.

Scheduled Maintenance Date:

Monday, May 18, 2026

Time:

9:00 – 18:00

Affected Sites:

SR1, IK1, YH1, AW1, IZ1 (all PFCP sites)

Planned Maintenance:

1. Kubernetes cluster upgrade: Running applications and Workspace execution environments will be stopped and recreated.
2. Flux version update: Deprecated Flux APIs will be removed and will no longer be available.
3. Database maintenance: The PFCP portal and Grafana will be temporarily unavailable, and the authentication system’s user group propagation will be paused.

Action requested of users:

• If you use Kubernetes services or Workspace, save any necessary data to persistent storage before maintenance begins.
• If you need to preserve in-progress computation state, complete checkpointing before maintenance starts.
• If you have created Pods directly without a higher-level resource such as a Deployment or Job, those Pods will not be recreated automatically. Please review your configuration in advance.
• If you use deprecated Flux APIs, please check whether you will be affected in advance.
• After maintenance, if expected groups are not reflected, please log in again.

1. Kubernetes Cluster Upgrade

Affected Service:

Kubernetes Service (including Workspace)

Reason:

To upgrade the Kubernetes clusters at all PFCP sites from version 1.34 to 1.35.

Impact (workloads):

• All nodes in the Kubernetes cluster will be upgraded sequentially. Applications (Pods) running on Kubernetes and Workspace execution environments will be temporarily stopped, deleted, and recreated.
• Pods created directly without a higher-level resource such as a Deployment or Job will not be recreated automatically.
• Data stored in persistent storage (PVCs, etc.) is not affected. However, data not in persistent storage — including data inside Pods and Workspace execution environments — will be lost.
• In-memory data and in-progress computation state will not be preserved.
• The Kubernetes API may be temporarily unstable during maintenance, causing kubectl operations and Workspace use to fail.

Impact (compute nodes):

• Up to 25% (rounded down) of shared nodes will be temporarily unavailable.
• Dedicated Kubernetes nodes allocated to users will also be upgraded sequentially; each node will be temporarily unavailable during its upgrade.
• If you have fewer than 4 dedicated nodes, they will be upgraded one at a time.
• If you have 4 or more dedicated nodes, up to 25% (rounded down) will be temporarily unavailable at a time.
• If you have only 1 dedicated node, it will be temporarily unavailable.

2. Flux Version Update

Affected Service:

Kubernetes Service

Reason:

To update the Flux version.

Impact:

• Deprecated Flux APIs will be removed and will no longer be available.
• For the list of affected APIs, see https://github.com/fluxcd/flux2/discussions/5572.

3. Database Maintenance

Affected Service:

PFCP Portal, Grafana, authentication system (user group propagation)

Reason:

Database maintenance.

Impact:

• The PFCP portal and Grafana will be temporarily unavailable.
• The authentication system’s user group propagation will be paused during maintenance.
• Login will still be possible, but expected groups may not be reflected.

For our maintenance policy, see the User Guide.

For details, see the User Guide.

Scheduled Date:

May 18, 2026

Planned Work:

Kubernetes cluster upgrades are planned for all clusters at all sites. PFCP performs regular Kubernetes minor version upgrades to improve cluster stability, performance, and security.

Details of any additional work will be announced at least two weeks before the maintenance date, in accordance with our regular maintenance policy.

Note:

The regular scheduled maintenance date is the first Monday of each month. This announcement is being made because the date differs from the usual schedule. No further notice will be sent if the date remains unchanged.

For our maintenance policy, see the User Guide.

This is an extremely serious incident: the malicious code can execute simply by having the package installed. To ensure the security of the entire cluster, PFCP has taken emergency security measures.

We apologize for the inconvenience. Please review the following information and take the necessary action (re-login) as described below.

Security measures taken:

• Action: Forced reset of kubectl credentials (tokens)
• Time: March 25, 2026, 13:00 JST

As a result of this action, all existing credentials have been invalidated. You will need to re-login before using kubectl going forward.

Impact:

• All existing kubectl credentials have been invalidated.
• Authentication errors will occur when running kubectl commands.
• Running workloads are not affected.

How to re-login:

Please follow the steps in the documentation below to log in again.

https://docs.pfcomputing.com/cluster/connect-to-cluster.html

Background (overview of the litellm compromise):

Versions 1.82.7 and 1.82.8 of “litellm” contained code that exfiltrated sensitive information—including environment variables and Kubernetes configuration (~/.kube/config)—to an external server.

The malicious code executes simply by starting a Python process. Because Kubernetes tokens may have been stolen without your knowledge, PFCP performed a cluster-wide reset.

Action requested of users:

1. Immediately check whether the affected version is installed in your PFCP execution environment and on your local PC.
   • Check command: pip list | grep litellm
2. Treat all secrets (SSH keys, cloud service API keys, etc.) accessible from any environment where this package was installed as potentially compromised, and revoke and reissue them.

Detailed information (GitHub Issue):

https://github.com/BerriAI/litellm/issues/24512

We apologize for the inconvenience and appreciate your understanding and cooperation as we take these necessary steps to maintain security.

For details, see the User Guide.

Figure 1: Browse view for resources such as Pods and Deployments

Figure 2: Manifest editing (left panel) and detailed information display including Prometheus metrics (right panel)

You can check the list of currently available container images and tags at https://console.pfcomputing.com/images.

The main updates in this release are as follows.

• Compiler performance improvements, bug fixes, and version updates for various dependency libraries
• Added the Migration Guide (MLSDK)
• Expanded MLSDK examples (/opt/pfn/pfcomp/codegen/MLSDK/examples/)

In the PFCP > HNC / HRQ / Tenant Resource Hard Limit & Usage dashboard, you can view limits and usage for Hierarchical Resource Quota.

For details, see the User Guide.

The English version of the user guide will also be available soon.

Mitigation Details:

ingress-nginx version upgrade (v1.14.3)

Mitigation Completion Date:

2026/02/04

User Impact:

No action is required on the user side. You can continue using the service as usual.

Vulnerability Details:

• CVE-2026-1580: https://github.com/kubernetes/kubernetes/issues/136677
• CVE-2026-24512: https://github.com/kubernetes/kubernetes/issues/136678
• CVE-2026-24513: https://github.com/kubernetes/kubernetes/issues/136679
• CVE-2026-24514: https://github.com/kubernetes/kubernetes/issues/136680

PFCP will continue to provide timely security updates so users can use our services with confidence.

For details, see the User Guide.

Start Time:

Around 2026/01/26 16:25

Resolution Time:

Around 2026/01/27 09:30

Affected Service:

Workspace feature of the Kubernetes service

Affected Sites:

All sites

Impact:

Workspaces created before the incident were stuck in “Preparing” status and could not be accessed.

Reason:

A defect in the Workspace feature

Date/Time:

Ongoing since around 2026/01/26 16:25

Affected Service:

Workspace feature of the Kubernetes service

Affected Sites:

All sites

Impact:

Workspaces created before the incident are stuck in “Preparing” status and cannot be accessed.

Reason:

A defect in the Workspace feature

Kubernetes has been upgraded to version 1.34. For details, see the official Kubernetes release blog post.

The software versions currently provided are as follows.

Cluster and Add-ons

Monitoring Service

• Software Versions - Preferred Computing Platform (PFCP) User Guide

PFCP performs regular version upgrades, including Kubernetes, to improve software stability, performance, and security. For details, see the PFCP User Guide / Maintenance Policy.

Scheduled Maintenance Date:

Monday, February 2, 2026

Time Window:

9:00-18:00

Affected Service:

Kubernetes Service (Persistent Storage)

Affected Site:

SR1

Reason:

• OS update for storage appliances

Impact:

• During the work window, response speed may experience slight delays.

For our maintenance policy, see the User Guide.

As a result, the following deprecated Flux API versions will no longer be available.

• source.toolkit.fluxcd.io/v1beta1
• source.toolkit.fluxcd.io/v1beta2
• kustomize.toolkit.fluxcd.io/v1beta1
• kustomize.toolkit.fluxcd.io/v1beta2
• helm.toolkit.fluxcd.io/v2beta1
• helm.toolkit.fluxcd.io/v2beta2
• image.toolkit.fluxcd.io/v1beta1
• notification.toolkit.fluxcd.io/v1beta1
• notification.toolkit.fluxcd.io/v1beta2

If you are using any of the APIs above, please migrate to the latest API versions. For details, see https://github.com/fluxcd/flux2/discussions/5572.

Scheduled Maintenance Date:

May 2026

Affected Service:

Kubernetes Service (Flux)

Affected Sites:

All sites

Impact:

• Deprecated Flux API versions will no longer be available.

For our maintenance policy, see the User Guide.

Scheduled Maintenance Date:

Monday, January 19, 2026

Time Window:

9:00-15:00

Affected Service:

Kubernetes Service

Affected Sites:

All sites

Reason:

• Upgrades for all Kubernetes clusters at all sites (from version 1.33 to 1.34)

Impact:

• The Kubernetes API may become temporarily unstable.
• Some user-allocated dedicated Kubernetes nodes may be temporarily unavailable during the upgrade.
  • If you have fewer than 4 contracted nodes, 1 node at a time may be unavailable.
  • If you have 4 or more contracted nodes, up to 25% (rounded down) may be unavailable.
  • If you have only 1 contracted node, no nodes will be available temporarily.

For our maintenance policy, see the User Guide.

For details, see the User Guide.

Because this is an experimental feature, the API schema may change in the future.

For details, see the User Guide.

Scheduled Date:

January 19, 2026

Planned Work:

We plan to upgrade all Kubernetes clusters at all sites. PFCP performs regular Kubernetes minor version upgrades to improve cluster stability, performance, and security.

Any additional maintenance items will be announced separately by two weeks before the maintenance date, in accordance with our regular maintenance policy.

Notes:

The regular scheduled maintenance date is the first Monday of each month. We are announcing this because the date differs from the usual schedule. If there is no date change, no schedule notice will be issued.

For our maintenance policy, see the User Guide.

For detailed usage, see File Storage and Shared Nodes.

For details on the vulnerabilities, please refer to the links below.

• runc version: 1.3.0 -> 1.3.3
• runc release notes: https://github.com/opencontainers/runc/releases/tag/v1.3.3
• CVE IDs
  • CVE-2025-31133 (CVSS v4: 7.3): https://github.com/opencontainers/runc/security/advisories/GHSA-9493-h29p-rfm2
  • CVE-2025-52881 (CVSS v4: 7.3): https://github.com/opencontainers/runc/security/advisories/GHSA-cgrx-mc8f-2prm
  • CVE-2025-52565 (CVSS v4: 7.3): https://github.com/opencontainers/runc/security/advisories/GHSA-qw9x-cqr3-wc7r

For details, see the User Guide.

Scheduled Date:

November 10, 2025

Planned Work:

Details will be announced separately by two weeks before the maintenance date, in accordance with our regular maintenance policy.

Notes:

The regular scheduled maintenance date is the first Monday of each month. We are announcing this because the date differs from the usual schedule. If there is no date change, no schedule notice will be issued.

For our maintenance policy, see the User Guide.

For details, see the User Guide.

Kubernetes has been upgraded to version 1.33. For details, see the official Kubernetes release blog post.

The software versions currently provided are as follows.

Cluster and Add-ons

Monitoring Service

• Software Versions - Preferred Computing Platform (PFCP) User Guide

PFCP performs regular version upgrades, including Kubernetes, to improve software stability, performance, and security. For details, see the PFCP User Guide / Maintenance Policy.

Scheduled Maintenance Date:

Monday, September 8, 2025

Time Window:

9:00-15:00

Affected Service:

Kubernetes Service

Affected Sites:

All sites

Reason:

• Upgrades for all Kubernetes clusters at all sites (from version 1.32 to 1.33)

Impact:

• The Kubernetes API may become temporarily unstable.
• Some user-allocated dedicated Kubernetes nodes may be temporarily unavailable during the upgrade.
  • If you have fewer than 4 contracted nodes, 1 node at a time may be unavailable.
  • If you have 4 or more contracted nodes, up to 25% (rounded down) may be unavailable.
  • If you have only 1 contracted node, no nodes will be available temporarily.

Time:

9:00

Affected Service:

Kubernetes Service

Affected Sites:

All sites

Reason:

End of support for the deprecated experimenal-shared PriorityClass

Impact:

• New Pod creation using the experimenal-shared PriorityClass will fail.
• Please migrate to the already available shared-standard PriorityClass.
• For information on using shared nodes, see the User Guide.

For our maintenance policy, see the User Guide.

For details, see the User Guide.

Scheduled Date:

September 8, 2025

Planned Work:

We plan to upgrade Kubernetes clusters at all sites. PFCP performs regular minor version upgrades to improve cluster stability, performance, and security.

Any additional maintenance items will be announced separately by two weeks before the maintenance date, in accordance with our regular maintenance policy.

Notes:

The regular scheduled maintenance date is the first Monday of each month. We are announcing this because the date differs from the usual schedule. If there is no date change, no schedule notice will be issued.

For our maintenance policy, see the User Guide.

For details, see the User Guide.

• Importing cloud storage data into clusters - Preferred Computing Platform (PFCP) User Guide

This tutorial walks through a scenario where you configure identity federation with a public cloud and securely copy data stored in user-managed AWS S3 to a Persistent Volume on PFCP. Please give it a try.

Authentication is configured automatically for each published web API, and access requires a Kubernetes ServiceAccount token. For detailed usage, see the User Guide.

Kubernetes has been upgraded to version 1.32. For details, see the official Kubernetes release blog post.

The software versions currently provided are as follows.

Cluster and Add-ons

Monitoring Service

• Software Versions - Preferred Computing Platform (PFCP) User Guide

PFCP performs regular version upgrades, including Kubernetes, to improve software stability, performance, and security. For details, see the PFCP User Guide / Maintenance Policy.

Scheduled Maintenance Date:

Monday, June 9, 2025

Time Window:

9:00-15:00

Affected Service:

Kubernetes Service

Affected Sites:

SR1, IK1

Reason:

Upgrades for all Kubernetes clusters at all sites (from version 1.31 to 1.32)

Impact:

• The Kubernetes API may become temporarily unstable.
• Some user-allocated dedicated Kubernetes nodes may be temporarily unavailable during the upgrade.
  • If you have fewer than 4 contracted nodes, 1 node at a time may be unavailable.
  • If you have 4 or more contracted nodes, up to 25% (rounded down) may be unavailable.
  • If you have only 1 contracted node, no nodes will be available temporarily.

Time Window:

9:00-13:00

Affected Service:

Monitoring Service

Affected Sites:

All sites

Reason:

Database maintenance work.

Impact:

Grafana will be temporarily unavailable during maintenance.

For our maintenance policy, see the User Guide.