Wrong order on ordinary 2 column image

[harvetech]

`
from transformers import LayoutLMv3ForTokenClassification
#from v3.helpers import prepare_inputs, boxes2inputs, parse_logits

model = LayoutLMv3ForTokenClassification.from_pretrained("hantian/layoutreader")

list of [left, top, right, bottom], bboxes of spans, should be range from 0 to 1000

page_width = 612
page_height = 792

norm_boxes = []
for (x1, y1, x2, y2) in filtered_boxes:
nx1 = int(x1 / page_width * 1000)
ny1 = int(y1 / page_height * 1000)
nx2 = int(x2 / page_width * 1000)
ny2 = int(y2 / page_height * 1000)
norm_boxes.append([nx1, ny1, nx2, ny2])

inputs = boxes2inputs(norm_boxes)
inputs = prepare_inputs(inputs, model)
logits = model(**inputs).logits.cpu().squeeze(0)
orders = parse_logits(logits, len(norm_boxes))
print(orders)`

I am using this sample code on bbox extracted by yolo based model. The order is completey random. I dont know where I am making a mistake. Do you have a sample images and bbox data for testing ?

Thanks :)

[ppaanngggg]

could you share me screenshots of your test cases? If your input is document layouts, please use surya-order

[harvetech]

Yes my input is document layout. Can you please give me the link of Surya-order ? Thanks.

[harvetech]

secondly, for the spans you are using fitz ( PyMuPdf ). I think pdfplumber has more pervasive licence. Do you think this code is equivalent to what you do currently using fitz ?

`
import pdfplumber

with pdfplumber.open("minueru_beam/2501.17887v1.pdf") as pdf:
page = pdf.pages[0] # same as your "break" after first page
width = page.width
height = page.height

# pdfplumber's closest equivalent to spans = words
words = page.extract_words()

boxes = []
for w in words:
    # pdfplumber gives x0, top, x1, bottom
    bbox = [w["x0"], w["top"], w["x1"], w["bottom"]]
    boxes.append(bbox)

print("page size:", width, height)
print("num boxes:", len(boxes))
print(boxes[:10])

`

btw it does not work with these boxes from pdfplumber it only works if the boxes are calculated via pymupdf. Any input ?

[ppaanngggg]

Yes my input is document layout. Can you please give me the link of Surya-order ? Thanks.

use this https://huggingface.co/vikp/surya_order , but you need to install a older version of surya

[ppaanngggg]

secondly, for the spans you are using fitz ( PyMuPdf ). I think pdfplumber has more pervasive licence. Do you think this code is equivalent to what you do currently using fitz ?

` import pdfplumber

with pdfplumber.open("minueru_beam/2501.17887v1.pdf") as pdf: page = pdf.pages[0] # same as your "break" after first page width = page.width height = page.height

# pdfplumber's closest equivalent to spans = words
words = page.extract_words()

boxes = []
for w in words:
    # pdfplumber gives x0, top, x1, bottom
    bbox = [w["x0"], w["top"], w["x1"], w["bottom"]]
    boxes.append(bbox)

print("page size:", width, height)
print("num boxes:", len(boxes))
print(boxes[:10])

`

btw it does not work with these boxes from pdfplumber it only works if the boxes are calculated via pymupdf. Any input ?

inputs should be something like lines. You may need to adjust params accoridng to https://github.com/jsvine/pdfplumber?tab=readme-ov-file#extracting-text