JackrabbitLabs-Releases/CSE-release
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

RC1

Browse Source
This commit is contained in:
Grant Mackey 2024-04-08 06:22:45 +00:00
parent 896b8819b8
commit aa92fdbdd1
14 changed files with 447 additions and 1248 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
LICENSE
View File

@ -58,7 +58,7 @@ APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.
Copyright 2024 Jackrabbit-Labs-LLC
Copyright 2024 Jackrabbit-Founders-LLC
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

23
Makefile
View File

@ -13,18 +13,20 @@
# ******************************************************************************
CC=gcc
CFLAGS= -g3 -O0 -Wall -Wextra
MACROS=-D CSE_VERBOSE
INCLUDE_DIR=/usr/local/include
LIB_DIR=/usr/local/lib
INCLUDE_PATH=-I $(INCLUDE_DIR) -I /usr/include/glib-2.0 -I /usr/lib/`uname -m`-linux-gnu/glib-2.0/include/ -I /usr/lib64/glib-2.0/include
LIB_PATH=-L $(LIB_DIR)
LIBS=-l yamlloader -l yaml -l glib-2.0 -l mctp -l uuid -l ptrqueue -l fmapi -l emapi -l arrayutils -l pciutils -l timeutils -l pci
CFLAGS?= -g3 -O0 -Wall -Wextra
MACROS?=-D CSE_VERBOSE
INCLUDE_DIR?=/usr/local/include
LIB_DIR?=/usr/local/lib
LOCAL_INCLUDE_DIR?=./include
LOCAL_LIB_DIR?=./lib
INCLUDE_PATH=-I $(LOCAL_INCLUDE_DIR) -I $(INCLUDE_DIR) -I /usr/include/glib-2.0 -I /usr/lib/`uname -m`-linux-gnu/glib-2.0/include/ -I /usr/lib64/glib-2.0/include
LIB_PATH=-L $(LOCAL_LIB_DIR) -L $(LIB_DIR)
LIBS=-l yamlloader -l yaml -l glib-2.0 -l mctp -l uuid -l ptrqueue -l fmapi -l emapi -l arrayutils -l timeutils -l pci -l cxlstate -l pciutils
TARGET=cse
all: $(TARGET)
$(TARGET): main.c options.o state.o signals.o emapi_handler.o fmapi_handler.o fmapi_isc_handler.o fmapi_psc_handler.o fmapi_vsc_handler.o fmapi_mpc_handler.o fmapi_mcc_handler.o
$(TARGET): main.c options.o state.o signals.o emapi_handler.o fmapi_handler.o fmapi_isc_handler.o fmapi_psc_handler.o fmapi_vsc_handler.o fmapi_mpc_handler.o fmapi_mcc_handler.o
$(CC) $^ $(CFLAGS) $(MACROS) $(INCLUDE_PATH) $(LIB_PATH) $(LIBS) -o $@
emapi_handler.o: emapi_handler.c emapi_handler.h
@ -66,8 +68,11 @@ doc:
install: $(TARGET)
sudo cp $(TARGET) /usr/local/bin/
uninstall:
sudo rm /usr/local/bin/$(TARGET)
# List all non file name targets as PHONY
.PHONY: all clean doc install
.PHONY: all clean doc install uninstall
# Variables
# $^ Will expand to be all the sensitivity list

119
README.md
View File

@ -1,22 +1,109 @@
# Install
# Overview
Install the following Linux Packages:
CXL Switch Emulator (CSE) is a software application that emulates the
management path functionality of a CXL switch. It is compliant with the CXL
Fabric Management API of the CXL 2.0 specification. All CXL FM management
commands are sent using MCTP.
- libyaml libyaml-dev doxygen uuid-dev pciutils-dev
CSE listens for connections from a Fabric Manager (FM) over TCP on a default
port of 2508. Only one remote connection is supported at a time. After a
connection is terminated by the remote Fabric Manager, CSE will wait for
another connection.
Clone the following repositories from code.jrlabs.io
# Supported Operating System Versions
- JackrabbitLabs/array_utils.git
- JackrabbitLabs/ptr_queue.git
- JackrabbitLabs/duplex_queue.git
- JackrabbitLabs/yaml_loader.git
- JackrabbitLabs/mctp.git
- JackrabbitLabs/fmapi.git
- JackrabbitLabs/cse.git
- JackrabbitLabs/jack.git
- Ubuntu 23.10
- Fedora 38, 39
On each repository perform the following commands (in order listed above)
> Note: Ubuntu 22.04 is not supported. This is due to some newer PCI features that
> are missing from the 5.15 Linux kernel that ships with Ubuntu 22.04.
# Building
1. Install OS libraries
Install the following build packages to compile the software on the following
operating systems.
**Ubuntu:**
```bash
apt install build-essential libglib2.0-dev libyaml-dev libpci-dev
```
**Fedora:**
```bash
```
2. Build Dependencies
To clone and build dependencies run:
```bash
./builddeps.bash
```
3. Build
After building the required dependencies run:
```bash
make
```
# Usage
1. Create host directory for memory mapped files
CSE primarily emulates the management path functionality of a CXL switch and
does not emulate PCIe/CXL data path functionality. However, some CXL FM API
management commands enable the Fabric Manager to read or write to the memory
space of CXL Type-3 devices. To emulate these commands, CSE can be configured
to instantiate memory mapped files as backing stores. This will allow a Fabric
Manager such as [Jack](https://github.com/JackrabbitLabs/jack) to write to an
arbitrary location in the memory space of a CXL Type-3 device, and then read
back the contents of those locations.
If the user does not wish to use the CXL LD CXL.io Memory Request commands,
this step can be skipped.
The config.yaml file has a parameter that specifies the location of a folder
in the host file system where the memory mapped files will be located. The
default location is `/cxl`. This directory must exist if the user plans to use
the CXL LD CXL.io commands.
While the target directory (e.g. /cxl) can reside on the root file system of
the host, it is recommended to use a tmpfs file system which can be created
with the following command:
```bash
sudo mkdir /cxl
sudo mount -t tmpfs -o size=4G,mode=1777 cxl /cxl
```
This should be performed *before* starting the CSE application.
The capacity of the tmpfs file system is dependent up on how much of the CXL
Type-3 device memory space needs to be written/read. The memory mapped files
are sparse allocated and will only consume the actually written capacity on
the tmpfs file system.
> Note: memory mapped files are only created for virtual CXL device profiles
> that have a `mmap: 1` in their profile definition in the config.yaml file.
2. Start the CSE application
The CSE application can be launched with the following command:
```bash
cse -lc config.yaml
```
The `-l` flag will configure CSE to produce log level output that states the
commands received and actions taken.
3. Exit
To exit the application, type `CTRL-C`.
- make
- make install

62
builddeps.bash Executable file
View File

@ -0,0 +1,62 @@
#!/bin/bash
# ******************************************************************************
#
# @file builddeps.bash
#
# @brief script to clone and build dependencies
#
# @copyright Copyright (C) 2024 Jackrabbit Founders LLC. All rights reserved.
#
# @date Apr 2024
# @author Barrett Edwards <code@jrlabs.io>
#
# ******************************************************************************
ARG=$1
CUR=`pwd`
HOST=https://github.com/
ORG=JackrabbitLabs
REPOS="array_utils time_utils pci_utils ptr_queue yaml_loader emapi fmapi cxl_state mctp"
SRC=src
INC=include
LIB=lib
if [ "${ARG}" = "clean" ] ; then
echo "Removing build dependencies"
rm -rf ${SRC}
rm -rf ${INC}
rm -rf ${LIB}
exit
fi
# Create source directory if not present
if [ ! -d ${SRC} ] ; then
mkdir ${SRC}
fi
# Create include directory if not present
if [ ! -d ${INC} ] ; then
mkdir ${INC}
fi
# Create lib directory if not present
if [ ! -d ${LIB} ] ; then
mkdir ${LIB}
fi
# Clone sub module repos into DST directory
for REPO in $REPOS ; do
echo "Cloning build dependencies"
git clone ${HOST}${ORG}/${REPO}.git ${SRC}/${REPO}
done
# Build sub modules
for REPO in $REPOS ; do
echo "Building dependencies"
cd ${SRC}/${REPO}/
make INCLUDE_DIR=${CUR}/${INC} LIB_DIR=${CUR}/${LIB} CFLAGS="-g3 -O0" install
cd ${CUR}
done
# Issue completion message
echo "Completed building dependencies"

48
emapi_handler.c
View File

@ -44,7 +44,7 @@
#include <ptrqueue.h>
#include <timeutils.h>
#include <emapi.h>
#include <cxlstate.h>
#include "signals.h"
#include "options.h"
@ -231,22 +231,22 @@ static int emop_conn_dev(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: EM API Connect Device. PPID: %d Device: %d\n", now, ppid, dev);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (ppid >= cxl_state->num_ports)
if (ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: PPID out of range. PPID: %d Total: %d\n", now, ppid, cxl_state->num_ports);
IFV(CLVB_ERRORS) printf("%s ERR: PPID out of range. PPID: %d Total: %d\n", now, ppid, cxls->num_ports);
goto send;
}
if (dev >= cxl_state->num_devices)
if (dev >= cxls->num_devices)
{
IFV(CLVB_ERRORS) printf("%s ERR: Device ID out of range. Device ID: %d Total: %d\n", now, dev, cxl_state->num_devices);
IFV(CLVB_ERRORS) printf("%s ERR: Device ID out of range. Device ID: %d Total: %d\n", now, dev, cxls->num_devices);
goto send;
}
if (cxl_state->devices[dev].name == NULL)
if (cxls->devices[dev].name == NULL)
{
IFV(CLVB_ERRORS) printf("%s ERR: Device is NULL. Device ID: %d\n", now, dev);
goto send;
@ -255,7 +255,7 @@ static int emop_conn_dev(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_ACTIONS) printf("%s ACT: Connecting Device %d to PPID %d\n", now, dev, ppid);
STEP // 10: Perform Action
state_connect_device(&cxl_state->ports[ppid], &cxl_state->devices[dev]);
cxls_connect(&cxls->ports[ppid], &cxls->devices[dev], cxls->dir);
STEP // 11: Prepare Response Object
@ -268,7 +268,7 @@ static int emop_conn_dev(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if(len < 0)
goto fail;
@ -370,21 +370,21 @@ static int emop_disconn_dev(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: EM API Disconnect Device. PPID: %d All: %d\n", now, ppid, all);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (all) {
start = 0;
end = cxl_state->num_ports;
end = cxls->num_ports;
}
else {
start = ppid;
end = ppid+1;
}
if (start >= cxl_state->num_ports)
if (start >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: PPID out of range. PPID: %d Total: %d\n", now, ppid, cxl_state->num_ports);
IFV(CLVB_ERRORS) printf("%s ERR: PPID out of range. PPID: %d Total: %d\n", now, ppid, cxls->num_ports);
goto send;
}
@ -392,12 +392,12 @@ static int emop_disconn_dev(struct mctp *m, struct mctp_action *ma)
for ( i = start ; i < end ; i++ )
{
// Validate if port is connected
if (cxl_state->ports[i].prsnt == 1)
if (cxls->ports[i].prsnt == 1)
{
IFV(CLVB_ACTIONS) printf("%s ACT: Disconnecting PPID %d\n", now, i);
// Perform disconnect
state_disconnect_device(&cxl_state->ports[i]);
cxls_disconnect(&cxls->ports[i]);
}
}
@ -412,7 +412,7 @@ static int emop_disconn_dev(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto fail;
@ -479,7 +479,7 @@ static int emop_list_dev(struct mctp *m, struct mctp_action *ma)
unsigned i, count;
__u8 num_requested, start_num;
struct cse_device *d;
struct cxl_device *d;
ENTER
@ -518,20 +518,20 @@ static int emop_list_dev(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: EM API list Devices. Start: %d Num: %d\n", now, start_num, num_requested);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (num_requested == 0)
num_requested = (cxl_state->num_devices - start_num);
num_requested = (cxls->num_devices - start_num);
if (start_num >= cxl_state->num_devices)
if (start_num >= cxls->num_devices)
{
IFV(CLVB_ERRORS) printf("%s ERR: Start num out of range. Start: %d Total: %d\n", now, start_num, num_requested);
goto send;
}
if ( (start_num + num_requested) >= cxl_state->num_devices)
num_requested = (cxl_state->num_devices - start_num);
if ( (start_num + num_requested) >= cxls->num_devices)
num_requested = (cxls->num_devices - start_num);
STEP // 10: Perform Action
IFV(CLVB_ACTIONS) printf("%s ACT: Responding with %d devices\n", now, num_requested);
@ -539,7 +539,7 @@ static int emop_list_dev(struct mctp *m, struct mctp_action *ma)
STEP // 11: Prepare Response Object
for ( i = 0 ; i < num_requested ; i++ )
{
d = &cxl_state->devices[start_num + i];
d = &cxls->devices[start_num + i];
// Serialize the id number
rspb->payload[len+0] = start_num + i;
@ -565,7 +565,7 @@ static int emop_list_dev(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
STEP // 15: Fill Response Header
ma->rsp->len = emapi_fill_hdr(&rspm.hdr, EMMT_RSP, reqm.hdr.tag, rc, reqm.hdr.opcode, len, count, 0);

2
fmapi_handler.c
View File

@ -46,8 +46,6 @@
#include "options.h"
#include "state.h"
#include <fmapi.h>
#include "fmapi_handler.h"

45
fmapi_isc_handler.c
View File

@ -43,6 +43,7 @@
#include <mctp.h>
#include <ptrqueue.h>
#include <timeutils.h>
#include <cxlstate.h>
#include "signals.h"
#include "options.h"
@ -154,18 +155,18 @@ int fmop_isc_bos(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API ISC Background Operation Status\n", now);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
STEP // 10: Perform Action
STEP // 11: Prepare Response Object
rsp.obj.isc_bos.running = cxl_state->bos_running;
rsp.obj.isc_bos.pcnt = cxl_state->bos_pcnt;
rsp.obj.isc_bos.opcode = cxl_state->bos_opcode;
rsp.obj.isc_bos.rc = cxl_state->bos_rc;
rsp.obj.isc_bos.ext = cxl_state->bos_ext;
rsp.obj.isc_bos.running = cxls->bos_running;
rsp.obj.isc_bos.pcnt = cxls->bos_pcnt;
rsp.obj.isc_bos.opcode = cxls->bos_opcode;
rsp.obj.isc_bos.rc = cxls->bos_rc;
rsp.obj.isc_bos.ext = cxls->bos_ext;
STEP // 12: Serialize Response Object
len = fmapi_serialize(rsp.buf->payload, &rsp.obj, fmapi_fmob_rsp(req.hdr.opcode));
@ -176,7 +177,7 @@ int fmop_isc_bos(struct mctp *m, struct mctp_action *ma)
//send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -268,19 +269,19 @@ int fmop_isc_id(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API ISC Identify\n", now);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
STEP // 10: Perform Action
STEP // 11: Prepare Response Object
rsp.obj.isc_id_rsp.vid = cxl_state->vid;
rsp.obj.isc_id_rsp.did = cxl_state->did;
rsp.obj.isc_id_rsp.svid = cxl_state->svid;
rsp.obj.isc_id_rsp.ssid = cxl_state->ssid;
rsp.obj.isc_id_rsp.sn = cxl_state->sn;
rsp.obj.isc_id_rsp.size = cxl_state->max_msg_size_n;
rsp.obj.isc_id_rsp.vid = cxls->vid;
rsp.obj.isc_id_rsp.did = cxls->did;
rsp.obj.isc_id_rsp.svid = cxls->svid;
rsp.obj.isc_id_rsp.ssid = cxls->ssid;
rsp.obj.isc_id_rsp.sn = cxls->sn;
rsp.obj.isc_id_rsp.size = cxls->max_msg_size_n;
STEP // 12: Serialize Response Object
len = fmapi_serialize(rsp.buf->payload, &rsp.obj, fmapi_fmob_rsp(req.hdr.opcode));
@ -291,7 +292,7 @@ int fmop_isc_id(struct mctp *m, struct mctp_action *ma)
//send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -383,14 +384,14 @@ int fmop_isc_msg_limit_get(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API ISC Get Response Message Limit\n", now);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
STEP // 10: Perform Action
STEP // 11: Prepare Response Object
rsp.obj.isc_msg_limit.limit = cxl_state->msg_rsp_limit_n;
rsp.obj.isc_msg_limit.limit = cxls->msg_rsp_limit_n;
STEP // 12: Serialize Response Object
len = fmapi_serialize(rsp.buf->payload, &rsp.obj, fmapi_fmob_rsp(req.hdr.opcode));
@ -401,7 +402,7 @@ int fmop_isc_msg_limit_get(struct mctp *m, struct mctp_action *ma)
//send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -493,7 +494,7 @@ int fmop_isc_msg_limit_set(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API ISC Set Response Message Limit\n", now);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (req.obj.isc_msg_limit.limit < 8 || req.obj.isc_msg_limit.limit > 20)
@ -503,10 +504,10 @@ int fmop_isc_msg_limit_set(struct mctp *m, struct mctp_action *ma)
}
STEP // 10: Perform Action
cxl_state->msg_rsp_limit_n = req.obj.isc_msg_limit.limit;
cxls->msg_rsp_limit_n = req.obj.isc_msg_limit.limit;
STEP // 11: Prepare Response Object
rsp.obj.isc_msg_limit.limit = cxl_state->msg_rsp_limit_n;
rsp.obj.isc_msg_limit.limit = cxls->msg_rsp_limit_n;
STEP // 12: Serialize Response Object
len = fmapi_serialize(rsp.buf->payload, &rsp.obj, fmapi_fmob_rsp(req.hdr.opcode));
@ -519,7 +520,7 @@ int fmop_isc_msg_limit_set(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
STEP // 15: Fill Response Header
ma->rsp->len = fmapi_fill_hdr(&rsp.hdr, FMMT_RESP, req.hdr.tag, req.hdr.opcode, 0, len, rc, 0);

21
fmapi_mcc_handler.c
View File

@ -43,6 +43,7 @@
#include <mctp.h>
#include <ptrqueue.h>
#include <timeutils.h>
#include <cxlstate.h>
#include "signals.h"
#include "options.h"
@ -108,7 +109,7 @@
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_get_ld_alloc(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_get_ld_alloc(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -217,7 +218,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_get_qos_alloc(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_get_qos_alloc(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -311,7 +312,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_get_qos_ctrl(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_get_qos_ctrl(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -404,7 +405,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_get_qos_limit(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_get_qos_limit(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -498,7 +499,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_get_qos_stat(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_get_qos_stat(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -584,7 +585,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_info(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_info(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -673,7 +674,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_set_ld_alloc(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_set_ld_alloc(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -795,7 +796,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_set_qos_alloc(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_set_qos_alloc(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -904,7 +905,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_set_qos_ctrl(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_set_qos_ctrl(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];
@ -1006,7 +1007,7 @@ end:
* 11: Serialize Response Header
* 12: Return length of MF API Message (FMLN_HDR + object)
*/
int fmop_mcc_set_qos_limit(struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
int fmop_mcc_set_qos_limit(struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp)
{
INIT
char now[ISO_TIME_BUF_LEN];

51
fmapi_mpc_handler.c
View File

@ -44,6 +44,7 @@
#include <ptrqueue.h>
#include <timeutils.h>
#include <arrayutils.h>
#include <cxlstate.h>
#include "signals.h"
#include "options.h"
@ -82,16 +83,16 @@
/* PROTOTYPES ================================================================*/
int fmop_mcc_get_ld_alloc (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_alloc (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_ctrl (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_limit (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_stat (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_info (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_ld_alloc (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_qos_alloc (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_qos_ctrl (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_qos_limit (struct port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_ld_alloc (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_alloc (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_ctrl (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_limit (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_get_qos_stat (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_info (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_ld_alloc (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_qos_alloc (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_qos_ctrl (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
int fmop_mcc_set_qos_limit (struct cxl_port *p, struct fmapi_msg *req, struct fmapi_msg *rsp);
/* GLOBAL VARIABLES ==========================================================*/
@ -133,7 +134,7 @@ int fmop_mpc_cfg(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct port *p;
struct cxl_port *p;
__u16 reg;
ENTER
@ -170,17 +171,17 @@ int fmop_mpc_cfg(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API MPC LD CXL.io Config. PPID: %d LDID: %d\n", now, req.obj.mpc_cfg_req.ppid, req.obj.mpc_cfg_req.ldid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
// Validate port number
if (req.obj.mpc_cfg_req.ppid >= cxl_state->num_ports)
if (req.obj.mpc_cfg_req.ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: Invalid Port number requested. PPID: %d\n", now, req.obj.mpc_cfg_req.ppid);
goto send;
}
p = &cxl_state->ports[req.obj.mpc_cfg_req.ppid];
p = &cxls->ports[req.obj.mpc_cfg_req.ppid];
// Validate port is not bound
//if ( !(p->state == FMPS_DISABLED) )
@ -254,7 +255,7 @@ int fmop_mpc_cfg(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -313,7 +314,7 @@ int fmop_mpc_mem(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct port *p;
struct cxl_port *p;
__u64 base, max, ld_size, granularity;
ENTER
@ -350,17 +351,17 @@ int fmop_mpc_mem(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API MPC LD CXL.io Mem. PPID: %d LDID: %d\n", now, req.obj.mpc_mem_req.ppid, req.obj.mpc_mem_req.ldid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
// Validate port number
if (req.obj.mpc_mem_req.ppid >= cxl_state->num_ports)
if (req.obj.mpc_mem_req.ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: Invalid Port number requested. PPID: %d\n", now, req.obj.mpc_mem_req.ppid);
goto send;
}
p = &cxl_state->ports[req.obj.mpc_mem_req.ppid];
p = &cxls->ports[req.obj.mpc_mem_req.ppid];
// Validate port is not bound
//if ( !(p->state == FMPS_DISABLED) )
@ -453,7 +454,7 @@ int fmop_mpc_mem(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -509,7 +510,7 @@ int fmop_mpc_tmc(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct port *p;
struct cxl_port *p;
ENTER
STEP // 1: Initialize variables
@ -544,7 +545,7 @@ int fmop_mpc_tmc(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API MPC Tunneled Management Command. PPID: %d\n", now, req.obj.mpc_tmc_req.ppid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
@ -556,12 +557,12 @@ int fmop_mpc_tmc(struct mctp *m, struct mctp_action *ma)
}
// Validate port number
if (req.obj.mpc_tmc_req.ppid >= cxl_state->num_ports)
if (req.obj.mpc_tmc_req.ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s Invalid Port number requested. PPID: %d\n", now, req.obj.mpc_tmc_req.ppid);
goto send;
}
p = &cxl_state->ports[req.obj.mpc_tmc_req.ppid];
p = &cxls->ports[req.obj.mpc_tmc_req.ppid];
// Validate device attached to port is an MLD port
if ( !(p->dt == FMDT_CXL_TYPE_3 || p->dt == FMDT_CXL_TYPE_3_POOLED) )
@ -637,7 +638,7 @@ sub:
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;

94
fmapi_psc_handler.c
View File

@ -43,6 +43,7 @@
#include <mctp.h>
#include <ptrqueue.h>
#include <timeutils.h>
#include <cxlstate.h>
#include "signals.h"
#include "options.h"
@ -121,7 +122,7 @@ int fmop_psc_cfg(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct port *p;
struct cxl_port *p;
__u16 reg;
ENTER
@ -158,15 +159,15 @@ int fmop_psc_cfg(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API PSC CXL.io Config. PPID: %d\n", now, req.obj.psc_cfg_req.ppid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (req.obj.psc_cfg_req.ppid >= cxl_state->num_ports)
if (req.obj.psc_cfg_req.ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: Requested PPDI exceeds number of ports present. Requested PPID: %d Present: %d\n", now, req.obj.psc_cfg_req.ppid, cxl_state->num_ports);
IFV(CLVB_ERRORS) printf("%s ERR: Requested PPDI exceeds number of ports present. Requested PPID: %d Present: %d\n", now, req.obj.psc_cfg_req.ppid, cxls->num_ports);
goto send;
}
p = &cxl_state->ports[req.obj.psc_cfg_req.ppid];
p = &cxls->ports[req.obj.psc_cfg_req.ppid];
// Validate port is not bound or is an MLD port
//if ( !(p->state == FMPS_DISABLED || p->ld > 0) )
@ -222,7 +223,7 @@ int fmop_psc_cfg(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -315,14 +316,45 @@ int fmop_psc_id(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API PSC Identify Switch Device\n", now);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
STEP // 10: Perform Action
STEP // 11: Prepare Response Object
state_conv_identity(cxl_state, &rsp.obj.psc_id_rsp);
{
struct cxl_switch *cs = cxls;
struct fmapi_psc_id_rsp *fi = &rsp.obj.psc_id_rsp;
// Zero out destination
memset(fi, 0, sizeof(*fi));
// Copy static information
fi->ingress_port = cs->ingress_port; //!< Ingress Port ID
fi->num_ports = cs->num_ports; //!< Total number of physical ports
fi->num_vcss = cs->num_vcss; //!< Max number of VCSs
fi->num_vppbs = cs->num_vppbs; //!< Max number of vPPBs
fi->num_decoders = cs->num_decoders; //!< Number of HDM decoders available per USP
// Compute dynamic information
for ( int i = 0 ; i < cs->num_ports ; i++ ) {
if ( cs->ports[i].state != FMPS_DISABLED )
fi->active_ports[i/8] |= (0x01 << (i % 8));
}
for ( int i = 0 ; i < cs->num_vcss ; i++ ) {
if ( cs->vcss[i].state == FMVS_ENABLED)
fi->active_vcss[i/8] |= (0x01 << (i % 8));
}
for ( int i = 0 ; i < cs->num_vcss ; i++ ) {
for ( int j = 0 ; j < MAX_VPPBS_PER_VCS ; j++ ) {
if ( cs->vcss[i].vppbs[j].bind_status != FMBS_UNBOUND )
fi->active_vppbs++;
}
}
}
STEP // 12: Serialize Response Object
len = fmapi_serialize(rsp.buf->payload, &rsp.obj, fmapi_fmob_rsp(req.hdr.opcode));
@ -333,7 +365,7 @@ int fmop_psc_id(struct mctp *m, struct mctp_action *ma)
//send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -429,7 +461,7 @@ int fmop_psc_port(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API PSC Get Physical Port Status. Num: %d\n", now, req.obj.psc_port_req.num);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
@ -441,11 +473,35 @@ int fmop_psc_port(struct mctp *m, struct mctp_action *ma)
id = req.obj.psc_port_req.ports[i];
// Validate portid
if (id >= cxl_state->num_ports)
if (id >= cxls->num_ports)
continue;
// Copy the data
state_conv_port_info(&cxl_state->ports[id], &rsp.obj.psc_port_rsp.list[i]);
struct cxl_port *src = &cxls->ports[id];
struct fmapi_psc_port_info *dst = &rsp.obj.psc_port_rsp.list[i];
// Zero out destination
memset(dst, 0, sizeof(*dst));
// Copy static information
dst->ppid = src->ppid; //!< Physical Port ID
dst->state = src->state; //!< Current Port Configuration State [FMPS]
dst->dv = src->dv; //!< Connected Device CXL Version [FMDV]
dst->dt = src->dt; //!< Connected Device Type [FMDT]
dst->cv = src->cv; //!< Connected device CXL Version [FMCV]
dst->mlw = src->mlw; //!< Max link width
dst->nlw = src->nlw; //!< Negotiated link width [FMNW]
dst->speeds = src->speeds; //!< Supported Link speeds vector [FMSS]
dst->mls = src->mls; //!< Max Link Speed [FMMS]
dst->cls = src->cls; //!< Current Link Speed [FMMS]
dst->ltssm = src->ltssm; //!< LTSSM State [FMLS]
dst->lane = src->lane; //!< First negotiated lane number
dst->lane_rev = src->lane_rev;//!< Link State Flags [FMLF] and [FMLO]
dst->perst = src->perst; //!< Link State Flags [FMLF] and [FMLO]
dst->prsnt = src->prsnt; //!< Link State Flags [FMLF] and [FMLO]
dst->pwrctrl = src->pwrctrl; //!< Link State Flags [FMLF] and [FMLO]
dst->num_ld = src->ld; //!< Supported Logical Device (LDs) count
rsp.obj.psc_port_rsp.num++;
}
@ -458,7 +514,7 @@ int fmop_psc_port(struct mctp *m, struct mctp_action *ma)
//send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -517,7 +573,7 @@ int fmop_psc_port_ctrl(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct port *p;
struct cxl_port *p;
ENTER
@ -553,15 +609,15 @@ int fmop_psc_port_ctrl(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API PSC Physical Port Control. PPID: %d Opcode: %d\n", now, req.obj.psc_port_ctrl_req.ppid, req.obj.psc_port_ctrl_req.opcode);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (req.obj.psc_port_ctrl_req.ppid >= cxl_state->num_ports)
if (req.obj.psc_port_ctrl_req.ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: Requested PPID exceeds number of ports present. Requested PPID: %d Present: %d\n", now, req.obj.psc_port_ctrl_req.ppid, cxl_state->num_ports);
IFV(CLVB_ERRORS) printf("%s ERR: Requested PPID exceeds number of ports present. Requested PPID: %d Present: %d\n", now, req.obj.psc_port_ctrl_req.ppid, cxls->num_ports);
goto send;
}
p = &cxl_state->ports[req.obj.psc_port_ctrl_req.ppid];
p = &cxls->ports[req.obj.psc_port_ctrl_req.ppid];
STEP // 10: Perform Action
switch (req.obj.psc_port_ctrl_req.opcode)
@ -599,7 +655,7 @@ int fmop_psc_port_ctrl(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;

83
fmapi_vsc_handler.c
View File

@ -43,6 +43,7 @@
#include <mctp.h>
#include <ptrqueue.h>
#include <timeutils.h>
#include <cxlstate.h>
#include "signals.h"
#include "options.h"
@ -121,7 +122,7 @@ int fmop_vsc_aer(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct vcs *v;
struct cxl_vcs *v;
ENTER
@ -157,15 +158,15 @@ int fmop_vsc_aer(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API VSC Generate AER Event. VCSID: %d vPPBID: %d\n", now, req.obj.vsc_aer_req.vcsid, req.obj.vsc_aer_req.vppbid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
if (req.obj.vsc_aer_req.vcsid >= cxl_state->num_vcss)
if (req.obj.vsc_aer_req.vcsid >= cxls->num_vcss)
{
IFV(CLVB_ERRORS) printf("%s ERR: Requested VCSID exceeds number of VCSs present. Requested VCSID: %d Present: %d\n", now, req.obj.vsc_aer_req.vcsid, cxl_state->num_vcss);
IFV(CLVB_ERRORS) printf("%s ERR: Requested VCSID exceeds number of VCSs present. Requested VCSID: %d Present: %d\n", now, req.obj.vsc_aer_req.vcsid, cxls->num_vcss);
goto send;
}
v = &cxl_state->vcss[req.obj.vsc_aer_req.vcsid];
v = &cxls->vcss[req.obj.vsc_aer_req.vcsid];
// Validate vppbid
if (req.obj.vsc_aer_req.vppbid >= v->num)
@ -188,7 +189,7 @@ int fmop_vsc_aer(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -247,9 +248,9 @@ int fmop_vsc_bind(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct vcs *v;
struct vppb *b;
struct port *p;
struct cxl_vcs *v;
struct cxl_vppb *b;
struct cxl_port *p;
ENTER
@ -285,20 +286,20 @@ int fmop_vsc_bind(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API VSC Bind vPPB. VCSID: %d vPPBID: %d PPID: %d LDID: 0x%04x\n", now, req.obj.vsc_bind_req.vcsid, req.obj.vsc_bind_req.vppbid, req.obj.vsc_bind_req.ppid, req.obj.vsc_bind_req.ldid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
// Validate vcsid
if (req.obj.vsc_bind_req.vcsid >= cxl_state->num_vcss)
if (req.obj.vsc_bind_req.vcsid >= cxls->num_vcss)
{
IFV(CLVB_ERRORS) printf("%s ERR: VCS ID out of range. VCSID: %d\n", now, req.obj.vsc_bind_req.vcsid);
goto send;
}
v = &cxl_state->vcss[req.obj.vsc_bind_req.vcsid];
v = &cxls->vcss[req.obj.vsc_bind_req.vcsid];
// Validate vppbid
if (req.obj.vsc_bind_req.vppbid >= cxl_state->vcss[req.obj.vsc_bind_req.vcsid].num)
if (req.obj.vsc_bind_req.vppbid >= cxls->vcss[req.obj.vsc_bind_req.vcsid].num)
{
IFV(CLVB_ERRORS) printf("%s ERR: vPPB ID out of range. vPPBID: %d\n", now, req.obj.vsc_bind_req.vppbid);
goto send;
@ -306,12 +307,12 @@ int fmop_vsc_bind(struct mctp *m, struct mctp_action *ma)
b = &v->vppbs[req.obj.vsc_bind_req.vppbid];
// Validate port id
if (req.obj.vsc_bind_req.ppid >= cxl_state->num_ports)
if (req.obj.vsc_bind_req.ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: PPID ID out of range. PPID: %d\n", now, req.obj.vsc_bind_req.ppid);
goto send;
}
p = &cxl_state->ports[req.obj.vsc_bind_req.ppid];
p = &cxls->ports[req.obj.vsc_bind_req.ppid];
// Check bindability to this port
@ -371,11 +372,11 @@ int fmop_vsc_bind(struct mctp *m, struct mctp_action *ma)
p->state = FMPS_DSP;
// Update Background Operation Status
cxl_state->bos_running = 0;
cxl_state->bos_pcnt = 100;
cxl_state->bos_opcode = req.hdr.opcode;
cxl_state->bos_rc = FMRC_SUCCESS;
cxl_state->bos_ext = 0;
cxls->bos_running = 0;
cxls->bos_pcnt = 100;
cxls->bos_opcode = req.hdr.opcode;
cxls->bos_rc = FMRC_SUCCESS;
cxls->bos_ext = 0;
STEP // 11: Prepare Response Object
@ -388,7 +389,7 @@ int fmop_vsc_bind(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -447,7 +448,7 @@ int fmop_vsc_info(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct vcs *v;
struct cxl_vcs *v;
unsigned i, k, stop, vppbid_start, vppbid_limit;
struct fmapi_vsc_info_blk *blk;
__u8 id;
@ -486,7 +487,7 @@ int fmop_vsc_info(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API VSC Get Virtual Switch Info. Num: %d\n", now, req.obj.vsc_info_req.num);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
@ -505,11 +506,11 @@ int fmop_vsc_info(struct mctp *m, struct mctp_action *ma)
break;
// Skip VCS IDs that exceed current size
if (id >= cxl_state->num_vcss)
if (id >= cxls->num_vcss)
continue;
// Get pointers to objects to copy
v = &cxl_state->vcss[id]; // The struct vcs to copy from
v = &cxls->vcss[id]; // The struct vcs to copy from
blk = &rsp.obj.vsc_info_rsp.list[i]; // The struct fmapi_vcs_info_blk to copy into
// Zero out destination
@ -546,7 +547,7 @@ int fmop_vsc_info(struct mctp *m, struct mctp_action *ma)
//send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;
@ -605,9 +606,9 @@ int fmop_vsc_unbind(struct mctp *m, struct mctp_action *ma)
unsigned rc;
int rv, len;
struct vcs *v;
struct vppb *b;
struct port *p;
struct cxl_vcs *v;
struct cxl_vppb *b;
struct cxl_port *p;
ENTER
@ -643,20 +644,20 @@ int fmop_vsc_unbind(struct mctp *m, struct mctp_action *ma)
IFV(CLVB_COMMANDS) printf("%s CMD: FM API VSC Unbind vPPB. VCSID: %d vPPBID: %d\n", now, req.obj.vsc_unbind_req.vcsid, req.obj.vsc_unbind_req.vppbid);
STEP // 8: Obtain lock on switch state
pthread_mutex_lock(&cxl_state->mtx);
pthread_mutex_lock(&cxls->mtx);
STEP // 9: Validate Inputs
// Validate vcsid
if (req.obj.vsc_unbind_req.vcsid >= cxl_state->num_vcss)
if (req.obj.vsc_unbind_req.vcsid >= cxls->num_vcss)
{
IFV(CLVB_ERRORS) printf("%s ERR: VCS ID out of range. VCSID: %d\n", now, req.obj.vsc_unbind_req.vcsid);
goto send;
}
v = &cxl_state->vcss[req.obj.vsc_unbind_req.vcsid];
v = &cxls->vcss[req.obj.vsc_unbind_req.vcsid];
// Validate vppbid
if (req.obj.vsc_unbind_req.vppbid >= cxl_state->vcss[req.obj.vsc_unbind_req.vcsid].num)
if (req.obj.vsc_unbind_req.vppbid >= cxls->vcss[req.obj.vsc_unbind_req.vcsid].num)
{
IFV(CLVB_ERRORS) printf("%s ERR: vPPB ID out of range. vPPBID: %d\n", now, req.obj.vsc_unbind_req.vppbid);
goto send;
@ -671,13 +672,13 @@ int fmop_vsc_unbind(struct mctp *m, struct mctp_action *ma)
}
// Validate port id that the vppb was bound to
if (b->ppid >= cxl_state->num_ports)
if (b->ppid >= cxls->num_ports)
{
IFV(CLVB_ERRORS) printf("%s ERR: PPID of bound port out of range. PPID: %d\n", now, b->ppid);
b->bind_status = FMBS_UNBOUND;
goto send;
}
p = &cxl_state->ports[b->ppid];
p = &cxls->ports[b->ppid];
// Check bindability to this port
@ -697,11 +698,11 @@ int fmop_vsc_unbind(struct mctp *m, struct mctp_action *ma)
b->ldid = 0;
// Update Background Operation Status
cxl_state->bos_running = 0;
cxl_state->bos_pcnt = 100;
cxl_state->bos_opcode = req.hdr.opcode;
cxl_state->bos_rc = FMRC_SUCCESS;
cxl_state->bos_ext = 0;
cxls->bos_running = 0;
cxls->bos_pcnt = 100;
cxls->bos_opcode = req.hdr.opcode;
cxls->bos_rc = FMRC_SUCCESS;
cxls->bos_ext = 0;
STEP // 11: Prepare Response Object
@ -714,7 +715,7 @@ int fmop_vsc_unbind(struct mctp *m, struct mctp_action *ma)
send:
STEP // 14: Release lock on switch state
pthread_mutex_unlock(&cxl_state->mtx);
pthread_mutex_unlock(&cxls->mtx);
if (len < 0)
goto end;

28
main.c
View File

@ -35,6 +35,7 @@
* mctp_run()
*/
#include <mctp.h>
#include <cxlstate.h>
#include "signals.h"
@ -65,6 +66,10 @@
#define IFV(u) if (opts[CLOP_VERBOSITY].u64 & u)
#define CSLN_PORTS 32
#define CSLN_VCSS 32
#define CSLN_VPPBS 256
/* ENUMERATIONS ==============================================================*/
/* STRUCTS ===================================================================*/
@ -97,6 +102,7 @@ int main(int argc, char* argv[])
struct mctp *m;
// Initialize varaibles
cxls = NULL;
stop_requested = 0;
rv = 1;
@ -112,8 +118,8 @@ int main(int argc, char* argv[])
signals_register();
STEP // 2: Initialize global state array
cxl_state = state_init(32, 32, 256);
if (cxl_state == NULL)
cxls = cxls_init(CSLN_PORTS, CSLN_VCSS, CSLN_VPPBS);
if (cxls == NULL)
{
printf("Error: state init failed \n");
goto end_options;
@ -122,7 +128,7 @@ int main(int argc, char* argv[])
STEP // 3: Load state file
if (opts[CLOP_CONFIG_FILE].set)
{
rv = state_load(cxl_state, opts[CLOP_CONFIG_FILE].str);
rv = state_load(cxls, opts[CLOP_CONFIG_FILE].str);
if (rv < 0)
{
printf("Error: state load config file failed \n");
@ -130,11 +136,13 @@ int main(int argc, char* argv[])
}
}
STEP // 4: Print the state
// STEP // 4: Build PCI Representation
STEP // 5: Print the state
if (opts[CLOP_PRINT_STATE].set)
state_print(cxl_state);
cxls_prnt(cxls);
STEP // 5: MCTP Init
STEP // 6: MCTP Init
m = mctp_init();
if (m == NULL)
goto end_state;
@ -150,7 +158,7 @@ int main(int argc, char* argv[])
// Set MCTP verbosity levels
mctp_set_verbosity(m, opts[CLOP_MCTP_VERBOSITY].u64);
STEP // 6: Run MCTP
STEP // 7: Run MCTP
rv = mctp_run(m, opts[CLOP_TCP_PORT].u16, opts[CLOP_TCP_ADDRESS].u32, MCRM_SERVER, 1, 1);
if (rv != 0)
{
@ -175,13 +183,13 @@ int main(int argc, char* argv[])
goto end_mctp;
}
STEP // 7: While loop
STEP // 8: While loop
while ( stop_requested == 0 )
{
sleep(1);
}
STEP // 8: Stop MCTP
STEP // 9: Stop MCTP
mctp_stop(m);
end_mctp:
@ -192,7 +200,7 @@ end_mctp:
end_state:
state_free(cxl_state);
cxls_free(cxls);
end_options:

926
state.c
View File

File diff suppressed because it is too large Load Diff

191
state.h
View File

@ -25,6 +25,7 @@
#include <pthread.h>
#include <fmapi.h>
#include <cxlstate.h>
/* MACROS ====================================================================*/
@ -45,198 +46,12 @@
/* STRUCTS ===================================================================*/
/**
* Multi Logical Device Object*
*
* This device aggregates all the descriptors for a CXL MLD Logical Device
*
* CXL 2.0 v1.0 Table 111,112,113,116,117,118,119
*/
struct mld {
/* LD Info: Table 111*/
__u64 memory_size; //!< Total device memory capacity
__u16 num; //!< Number of Logical Devices supported
__u8 epc; //!< Egress Port Congestion Supported
__u8 ttr; //!< Temporary Throughput Reduction Supported
/* LD Allocations: Table 112,113 */
__u8 granularity; //!< Memory Granularity [FMMG]
__u64 rng1[FM_MAX_NUM_LD]; //!< Range 1 Allocation Multiplier
__u64 rng2[FM_MAX_NUM_LD]; //!< Range 2 Allocation Multiplier
/* LD QoS Control parameters: Table 116*/
__u8 epc_en; //!< QoS Telem: Egress Port Congestion Enable. Bitfield [FMQT]
__u8 ttr_en; //!< QoS Telem: Temporary Throuhput Reduction Enable. Bitfield [FMQT]
__u8 egress_mod_pcnt; //!< Egress Moderate Percentage: Threshold in percent for Egress Port Congestion mechanism to indicate moderate congestion. Valid range is 1-100. Default is 10.
__u8 egress_sev_pcnt; //!< Egress Severe Percentage: Threshold in percent for Egress Port Congestion mechanism to indicate severe congestion. Valid range is 1-100. Default is 25
__u8 sample_interval; //!< Backpressure Sample Interval: Interval in ns for Egress Port Congestion mechanism to take samples. Valid range is 0-15. Default is 8 (800 ns of history). Value of 0 disables the mechanism.
__u16 rcb; //!< ReqCmpBasis. Estimated maximum sustained sum of requests and recent responses across the entire device, serving as the basis for QoS Limit Fraction. Valid range is 0-65,535. Value of 0 disables the mechanism. Default is 0.
__u8 comp_interval; //!< Completion Collection Interval: Interval in ns for Completion Counting mechanism to collect the number of transmitted responses in a single counter. Valid range is 0-255. Default is 64
/* LD QoS Status: Table 117*/
__u8 bp_avg_pcnt; //!< Backpressure Average Percentage: Current snapshot of the measured Egress Port average congestion. Table 117
/* LD QoS Allocated BW Fractions: Table 118 */
__u8 alloc_bw[FM_MAX_NUM_LD];
/* LD QoS BW Limit Fractions: Table 119 */
__u8 bw_limit[FM_MAX_NUM_LD];
__u8 *cfgspace[FM_MAX_NUM_LD]; //!< Buffers representing PCIe config space for each logical device
__u8 mmap; //!< Direction to mmap a file for the memory space
char *file; //!< Filename for mmaped file
__u8 *memspace; //!< Buffer representing memory space for entire logical device
};
/**
* Virtual PCIe-to-PCIe Bridge Object
*
* CXL 2.0 v1.0 Table 99
*/
struct vppb {
__u16 vppbid; //!< Index of this vPPB in the state->vppbs[] array
__u8 bind_status; //!< PBB Binding Status [FMBS]
__u8 ppid; //!< Physical port number of bound port
__u16 ldid; //!< ID of LD bound to port from MLD on associated physical port
};
/**
* Virtual CXL Switch Object
*
* CXL 2.0 v1.0 Table 99
*/
struct vcs {
__u8 vcsid; //!< VCS ID - Index of this vcs in the state->vcss[] array
__u8 state; //!< Virtual CXL switch State [FMVS]
__u8 uspid; //!< USP Physical Port ID
__u8 num; //!< Number of vPPBs
//!< Array of pointers to vPPB objects
struct vppb vppbs[MAX_VPPBS_PER_VCS];
};
/**
* CXL Switch Port Object
*
* CXL 2.0 v1.0 Table 92
*/
struct port {
__u8 ppid; //!< Port ID - Index of this port in the state->ports[] array
__u8 state; //!< Current Port Configuration State [FMPS]
__u8 dv; //!< Connected Device CXL version [FMDV]
__u8 dt; //!< Connected device type [FMDT]
__u8 cv; //!< Connected CXL version bitmask [FMVC]
__u8 mlw; //!< Max Link Width. Integer number of lanes (1,2,4,8,16)
__u8 nlw; //!< Negotiated Link Width [FMNW]
__u8 speeds; //!< Supported Link Speeds Vector [FMSS]
__u8 mls; //!< Maximum Link Speed [FMMS]
__u8 cls; //!< Current Link Speed [FMMS]
__u8 ltssm; //!< LTSSM State [FMLS]
__u8 lane; //!< First negotiated lane number (Integer lane number)
/** Link State Flags [FMLF] [FMLO] */
__u8 lane_rev; //!< Lane reversal state. 0=standard, 1=rev [FMLO]
__u8 perst; //!< PCIe Reset State PERST#
__u8 prsnt; //!< Port Presence pin state PRSNT#
__u8 pwrctrl; //!< Power Control State (PWR_CTRL)
__u8 ld; //!< Additional supported LD Count (beyond 1)
__u8 *cfgspace; //!< Buffer representing PCIe config space
struct mld *mld; //!< State for MLD
char *device_name; //!< Name of device used to populate this port
};
struct cse_device
{
char *name; //!< Name of device
__u8 rootport; //!< Root Port Device. 1=root, 2=endpoint
__u8 dv; //!< Connected Device CXL version [FMDV]
__u8 dt; //!< Connected device type [FMDT]
__u8 cv; //!< Connected CXL version bitmask [FMVC]
__u8 mlw; //!< Maximum Link Width. Integer number of lanes (1,2,4,8,16)
__u8 mls; //!< Maximum Link Speed [FMMS]
__u8 *cfgspace; //!< Buffer representing PCIe config space
struct mld *mld; //!< MLD info if this is an MLD
};
/**
* CXL Switch State Identify Information
*
* CXL 2.0 v1 Table 89
*/
struct cxl_switch_state {
__u8 version; //!< Device Management Version
__u16 vid; //!< PCIe Vendor ID
__u16 did; //!< PCIe Device ID
__u16 svid; //!< PCIe Subsystem Vendor ID
__u16 ssid; //!< PCIe Subsystem ID
__u64 sn; //!< Device Serial Number
__u8 max_msg_size_n; //!< Max fmapi msg size. 2^n
__u8 msg_rsp_limit_n; //!< Message Response Limit n of 2^n
__u8 bos_running; //!< Background operation status 0=none, 1=running
__u8 bos_pcnt; //!< Background operation percent complete [0-100]
__u16 bos_opcode; //!< Background operation opcode
__u16 bos_rc; //!< Background operation return code
__u16 bos_ext; //!< Background operation Extended Vendor Status
__u8 ingress_port; //!< Ingress Port ID
__u8 num_ports; //!< Total number of physical ports
__u8 num_vcss; //!< Max number of VCSs
__u16 num_vppbs; //!< Max number of vPPBs
__u16 active_vppbs; //!< Number of active vPPBs
__u8 num_decoders; //!< Number of HDM decoders available per USP
struct port *ports; //!< array of Port objects
struct vcs *vcss; //!< array of VCS objects
struct cse_device *devices; //!< array of device definitions
__u16 len_devices; //!< Number of entries supported in devices array
__u16 num_devices; //!< Number of entries in devices array
/* Port defaults */
__u8 mlw; //!< Max Link Width. Integer number of lanes (1,2,4,8,16)
__u8 speeds; //!< Supported Link Speeds Vector [FMSS]
__u8 mls; //!< Maximum Link Speed [FMMS]
char *dir; //!< Filepath to directory for instantiated memory
pthread_mutex_t mtx; //!< Mutex to control access to this object
};
/* PROTOTYPES ================================================================*/
struct cxl_switch_state *state_init(unsigned ports, unsigned vcss, unsigned vppbs);
int state_load(struct cxl_switch_state *s, char *filename);
void state_free(struct cxl_switch_state *s);
int state_connect_device(struct port *p, struct cse_device *d);
int state_disconnect_device(struct port *p);
/* Conversion Functions */
void state_conv_identity(struct cxl_switch_state *src, struct fmapi_psc_id_rsp *dst);
void state_conv_port_info(struct port *src, struct fmapi_psc_port_info *dst);
void state_conv_vcs_info(struct vcs *src, struct fmapi_vsc_info_blk *dst);
/* Print Functions */
void state_print(struct cxl_switch_state *s);
void state_print_identity(struct cxl_switch_state *s, unsigned indent);
void state_print_ports(struct cxl_switch_state *s, unsigned indent);
void state_print_port(struct port *p, unsigned indent);
void state_print_vcss(struct cxl_switch_state *s, unsigned indent);
void state_print_vcs(struct vcs *v, unsigned indent);
void state_print_vppb(struct vppb *b, unsigned indent);
void state_print_mld(struct mld *mld, unsigned indent);
void state_print_devices(struct cxl_switch_state *s);
int state_load(struct cxl_switch *s, char *filename);
/* GLOBAL VARIABLES ==========================================================*/
extern struct cxl_switch_state *cxl_state;
extern struct cxl_switch *cxls;
#endif //ifndef _STATE_H