CL-CLINT Register Handling to Support Multiple Clusters

[mridul-tosil]

The existing implementation of the CL-CLINT register handling in banshee-pulp is limited to a single cluster. Specifically, the code directly checks the x value against a single base address (self.engine.config.address.cl_clint) and its offset (+ 0x8) to determine whether it is a "set" or "clear" operation.

banshee/src/engine.rs

Line 837 in 0e14490

x if x >= self.engine.config.address.cl_clint

banshee/src/engine.rs

Line 1012 in 0e14490

x if x == self.engine.config.address.cl_clint => {

This implementation does not support systems with multiple clusters where each cluster has its own CLINT register mapped to different addresses.

Existing implementation

banshee/src/engine.rs

Line 837 in 0e14490

x if x >= self.engine.config.address.cl_clint

       x if x >= self.engine.config.address.cl_clint
            && x < self.engine.config.address.cl_clint + 0x8 =>
        {
            0
        }

Proposed implementation

        x if (0..self.engine.num_clusters).any(|i| {
            x >= (self.engine.config.address.cl_clint
                + self.engine.config.memory.periphs.offset * i as u32)
                && x <= (self.engine.config.address.cl_clint
                    + self.engine.config.memory.periphs.offset * i as u32
                    + 0x8) // Original size for cl_clint
        }) =>
        {
            0
        }

Existing implementation

banshee/src/engine.rs

Line 1012 in 0e14490

x if x == self.engine.config.address.cl_clint => {

        x if x == self.engine.config.address.cl_clint => {
            // clint set register
            let old_entry = self
                .cl_clint
                .fetch_or((value & mask) as usize, Ordering::SeqCst);
            // wake cores affected by this write
            let hart_base = (self.cluster_id * self.num_cores) as u32;
            for i in 0..32 {
                if ((!old_entry & (value & mask) as usize) & (1 << i)) != 0 {
                    trace!(
                        "  wakeup_wus.req[{}] from cluster-local CLINT",
                        (hart_base + i) as usize
                    );
                    self.wake((hart_base + i) as u32);
                }
            }
        }
        x if x == self.engine.config.address.cl_clint + 0x8 => {
            // clint clear register
            self.cl_clint
                .fetch_and(!(value & mask) as usize, Ordering::SeqCst);
        }

Proposed implementation

        x if (0..self.engine.num_clusters).any(|i| {
            let base = self.engine.config.address.cl_clint
                + self.engine.config.memory.periphs.offset * i as u32;
            x >= base && x <= base + 0x8
        }) =>
        {
            let id = (0..self.engine.num_clusters)
                .position(|i| {
                    let base = self.engine.config.address.cl_clint
                        + self.engine.config.memory.periphs.offset * i as u32;
                    addr >= base && addr <= base + 0x8
                })
                .unwrap();

            let offset = addr
                - (self.engine.config.address.cl_clint
                    + self.engine.config.memory.periphs.offset * id as u32);

            match offset {
                0 => {
                    // clint set register
                    let old_entry = self
                        .cl_clint
                        .fetch_or((value & mask) as usize, Ordering::SeqCst);
                    // wake cores affected by this write
                    let hart_base = (id * self.num_cores) as u32;
                    for i in 0..32 {
                        if ((!old_entry & (value & mask) as usize) & (1 << i)) != 0 {
                            trace!(
                                "  wakeup_wus.req[{}] from cluster-local CLINT",
                                (hart_base + i) as usize
                            );
                            self.wake((hart_base + i) as u32);
                        }
                    }
                }
                0x8 => {
                    // clint clear register
                    self.cl_clint
                        .fetch_and(!(value & mask) as usize, Ordering::SeqCst);
                }
                _ => (),
            }
        }

Additional Note:
I would like to mention that I am new to Rust programming and still learning the best practices. I apologize in advance if the code provided or the approach taken is not optimal.

@SamuelRiedel @paulsc96