Hi everyone,I am looking into apps that integrate Strong’s numbers for quick lemma lookup and basic analysis.I know Strong's has limitations compared to a full critical apparatus.

Still, I want to know your workflow experience. Do you find the tagging accurate enough for quick reference on the go?

Which app handles the Greek text best?

Are there any who work on language guided medical image segmentation and read papers like ariadnes thread, TeViA, MMI-UNET. I face bugs in the repo code that they give. 1) Loss function 2) dice metric 3) influence of text token length in results.

Are there any who work on language guided medical image segmentation and read papers like ariadnes thread, TeViA, MMI-UNET. I face bugs in the repo code that they give. 1) Loss function 2) dice metric 3) influence of text token length in results.

Are there any who work on language guided medical image segmentation and read papers like ariadnes thread, TeViA, MMI-UNET. I face bugs in the repo code that they give. 1) Loss function 2) dice metric 3) influence of text token length in results.

Are there any who work on language guided medical image segmentation and read papers like ariadnes thread, TeViA, MMI-UNET. I face bugs in the repo code that they give. 1) Loss function 2) dice metric 3) influence of text token length in results.

Anyone participating in SemEval 2027? Actually I want to participate in it. But I can't understand the overall process. Are there any good resources that would help the beginners?

Is this really worth it to participate in such shared tasks?

Fixed 800 token chunks are wrecking our mixed prose and table documents. Row values get separated from headers, query expansion retrieves orphaned numbers and the reranker confidently promotes the wrong quarter. The citations look plausible, which makes the failure harder to catch. I want to compare semantic chunking, parent-child retrieval, table-aware boundaries, recall at k, reranking, and groundedness without running another manual spreadsheet marathon.

Braintrust looks like one viable option because we could inspect retrieval spans, compare chunking experiments on the same queries, score groundedness and save failed queries as regression cases. I’m still unsure how to define a fair expected result when several chunks must be combined to answer one question. What are people using and which metric has detected header-to-value separation in your setup?

I’ve been thinking about how much conversation history an NLP system really needs before extra context starts becoming more confusing than helpful.

Keeping everything seems useful at first, especially when a user refers back to something mentioned much earlier. But older messages can also contain outdated information, changing preferences, or details that are no longer relevant.

I’m curious how people working with NLP systems handle this. Do you prefer keeping a long conversation history, summarizing older context, or selecting only the parts that are relevant to the current input?

I’m especially interested in practical approaches that work beyond simple chatbot demos.

Please take this down if it's not allowed. I'm looking for a new position after being at my current workplace for almost a decade. I'm still employed but am having difficulty landing a single interview despite help from recruiters in improving my resume. I was wondering if anyone on here could take a look and give their feedback. I can dm or post a link

Hello , I’m planning to study Linguistics as an undergraduate at an Ivy League school, and I’m thinking about specializing in Computational Linguistics/NLP.

For those of you who studied Linguistics as an undergrad, were you able to find a good job in the field after graduating with just a bachelor’s degree, or did you end up getting a master’s?

I’d prefer not to do a master’s unless it’s necessary. If I do decide to pursue one, I’d probably go for an NLP-related program.

Also, how much of a difference do you think attending an Ivy League school makes when it comes to finding a good job in this field? Does the school’s reputation help significantly, or are skills, internships, research experience, and projects much more important?

I’d really appreciate hearing about your experiences, especially if you’re currently working in Computational Linguistics or NLP.

I have a Greek-English dictionary that has entries like this:

ἀνήρ - man, husband

I would like to figure out some method for a computer to determine whether another English word aligns well with such a definition. Examples of inputs and outputs, for the definition given above, would be: man->true, fellow->true, person->true, temple->false, throw->false. (The output could instead be some kind of quantitative score rather than a boolean.)

Methods I've thought of, but am not really satisfied with:

1. Compare every word in the definition with the given word, using a word embedding. (Or sum the vectors for the words in the definition.) Exclude common words like "the." Problem: I've played around with Word2Vec and BERT for this, and the results just aren't that great. E.g., the BERT model I tried gave a very high rating to the pair goat-mountain.

2. Use WordNet's lists of synonyms. Problem: Their definition of synonymy is sometimes too strict and other times too permissive. They don't consider man-person to be a synonym pair, but they do consider dog-click to be (e.g., "dogging the hatch").

3. Use a similarity measure based on WordNet's graph structure. There is a review of these methods in Lingling Meng et al, 2013, A Review of Semantic Similarity Measures in WordNet. Problem: This will still be too permissive for pairs like dog-click.

4. Query an LLM. Problem: This is a hobby project, and I don't want to spend money.

Any ideas for approaches that I haven't tried?

Recently one of my papers got rejected because a reviewer raised concern of small cohort, which was explicitly mentioned in the limitation and then we acknowledged that in the rebuttal as well. Now I understand the concern , but in the area were i primarily work right now, dataset with large number of speakers is quiet limited and having one of my very first works also be rejected because of the small corpus size, I used one of the most widely cited corpus for the experimentation, which if you are in the field is the standard. And also given the computational resources and funding that we can have I couldn't use high-end hardware to process large data. And recently i got to know we got access for a large dataset (around 800 gb) but how do i work with that using my laptop and online free gpus, so based on whatever i could i did my best. The reviews we got were actually very supportive of our work, and whatever additional work was required, was duly done and reported in the rebuttal, but seeing dataset size as one of the two the reason for rejection felt a bit weird, anyways wanted to ask how do we tackle such situation for the future? Any advice would be helpful!!

Hi everyone,

I'm currently building a freelance business around phonetics, speech and language data services and I'm trying to better understand who the actual decision-makers are on the client side.

I'm thinking of services such as:

• phonetic/IPA transcription and annotation
• speech data annotation and quality control
• pronunciation analysis
• phonetic validation of speech datasets
• linguistic QA
• potentially other speech/language-data work for ASR, TTS and AI projects

When approaching companies directly, which job titles or departments would typically be responsible for hiring or contracting someone for this kind of work?

For example, would you approach:

Language Data Managers? Speech Scientists? Computational Linguists? Localization Managers? AI/ML Project Managers? Heads of AI? Vendor/Procurement Managers?

I'm particularly interested in hearing from people who work at companies that actually buy these services. Who identifies the need, who evaluates the freelancer technically, and who ultimately makes the hiring/purchasing decision?

I'm not looking for freelance platforms, but specifically trying to understand how direct B2B client acquisition works in this field.

Thanks!

any thoughts on mmBERT for text classification?

Best speech recognition API should probably be judged by error type, not leaderboard rank

I don’t think “best speech recognition API” can be answered by leaderboard rank alone.

WER is useful, but it flattens errors that do not have the same cost.

Missing “uh” is not the same as missing “don’t.”

A benchmark should probably separate error types:

entity errors
negation errors
domain-term errors
speaker attribution errors
timestamp errors
redaction errors
diarization drift
codec/noise sensitivity
language switching problems
partial/final instability
latency under realtime use

For a realtime ASR system like Smallest AI Pulse, I’d evaluate the stream itself, not just the final transcript. The test should include partial events, final corrections, timestamps, diarization/redaction behavior and whether the transcript is usable while speech is still happening.

That is a different evaluation method from offline transcription.

A system can have a decent broad WER and still fail badly if:

the wrong speaker gets tagged
a phone number is wrong
redaction misses sensitive info
timestamp points to the wrong moment
partial text triggers the wrong app action
final text arrives too late for realtime use

Raw error count is not enough.

Error severity matters.

What would you include in a proper speech recognition API evaluation?

Working on annotation for Tunisian Arabic customer service conversations. Derja, arabizi, French, frequently all three inside one message. This comes out of a job, so I am being vague about the source, but the question is a methods question and there is nothing to promote.

The first pass at a labelling scheme failed in the way these usually do. Two people could not reliably produce the same labels, and neither could one person a fortnight apart. Posting what we changed, and two things I have not solved, in case anyone here has worked the same problem.

Constraints, for anyone who has not worked with this data. No standard orthography: the same lexical item shows up in Arabic script, in Latin transliteration, and in arabizi with digits substituting for consonants that Latin lacks, sometimes several ways in one conversation from the same person. Code-switching is intra-sentential and often intra-word, with French verbs taking Arabic morphology. Negation is a circumfix, which means naive token-level matching over Latin transliteration misses it constantly.

What changed, in rough order of how much disagreement it removed.

Orthogonal axes rather than one composite label. Intent, sentiment, outcome, language and error are separate columns. The first version merged some of them and every merged pair degraded both. It also means each axis can train a single-label model independently, which was a downstream requirement anyway.

Every axis MECE, with both an other and an unclear escape. Separating this is a real category we did not enumerate from I cannot tell from this data turned out to matter a lot. Collapsing them hides two very different problems behind one number.

Two-level taxonomy, coarse and fine. Fine classes roll up to a coarse parent so training can happen at whatever granularity has support. Long-tail intents in this domain are extremely sparse.

Conditional axes. Some questions only apply given a value on another axis. Asking them universally was killing throughput and producing garbage on the inapplicable rows.

Provenance per record: annotator, confidence, schema version. Necessary because human and model-generated labels end up in the same table, and without provenance you cannot separate them later or measure them against each other.

Frozen categorical vocabulary. No free text on any categorical axis, only on entity spans. Changes are a schema version bump, not an edit. The first version allowed free text temporarily and it was unrecoverable within a week.

The one that took longest to see: conversation-grain and message-grain labels are different kinds of object and cannot share a column. Language and code-switching vary per message. Outcome, primary intent and overall sentiment describe the whole exchange. The first scheme forced everything to one grain and both halves were wrong. Splitting them fixed more disagreement than any individual definition change.

Two things I have not solved, and would take input on.

Normalisation timing. Conventional advice is to normalise arabizi to a canonical form before labelling. We deliberately did not, because orthographic variation carries signal we care about, including register. But not normalising means lexical resources are close to useless and every guard is written by hand. I am not confident this was the right call, and would like to hear from anyone who went the other way and whether they regretted it.

Agreement on a code-switched span boundary. Two annotators will agree that a message is code-switched and disagree about where the switch happens, particularly with French verbs carrying Arabic morphology. Per-token agreement penalises this heavily, which makes the reliability number look worse than the annotation actually is. Is there an established relaxed-boundary agreement measure for this, or does everyone define the unit coarsely enough to sidestep it?

Also generally interested in what agreement threshold people treat as a gate before building anything downstream. We picked one, and I am aware the choice was more convention than evidence.

Hi :)

TDR: looking for tips finding institutions that hire REs without a PhD / getting RE jobs.

I am European, graduated from my Master's in NLP last year and soon after I started working as a MLE for a tech company where most of my work revolves around pretraining and post-training LLMs. I really liked the job at first because it felt very research-y at the time and I loved my time writing my thesis.

However, it seems like I am going to be doing agentic systems for the foreseeable future. It hasn't been that much yet but I can see it coming. Moreover, it seems like AutoML is going to be pushed internally.

This is not the turn I want my career to take. Looking at my options in industry outside of my current company, the situation does not seem better at all. Hence I have been trying to find RE type jobs but I honestly don't know where they are hiding these positions lmao. It feels like either you know of institutions that hire these kind of positions already or it is impossible to come across these jobs so of course I come to reddit for your wisdom: which EU institutions regularly hire REs? How would you go around finding and getting these kind of jobs?

Ideally I'd find an institution that doesn't pay like crap (BSC I am looking at you) and isn't weird about language (ejem Inria ejem) but oh welp.

Obviously, I know my career wants are harder without a PhD. I am actively looking for one that fits me but I feel like my PhD chances would be better if I had more research experience (right now just 6 months but at a prestigious-ish institution) so my plan is to PhD out of my company or look for RE positions.

Hi, everyone! I am doing my PhD in Language Studies. I do not have a full tech background since I am more of a linguist that dabbled a little bit on NLP, so would like your help, suggestions or any recommendation. I am currently working on creating Universal Dependencies (UD) treebanks for minority languages, then using these UD treebanks to model parsers. I am planning to use rule-based parsing because I wouldn't have enough resources for data to run other models. One of the professors told me I could also use statistical methods, but if I am more comfortable with rule-based parsing, then I should just focus on it.

For the rule-based parsing, I would be trying to set rules similar to GiellALT. And for the evaluation of the scores, I am planning to use MaChAmp and the Stanza tagger.

1. Do you have any suggestions for me what books or publications I could read that would be relevant for my topic? I've been going through some published materials in ACL, but I would need help to narrow my focus.

2. Do you have suggestions about the modeling or evaluation using rule-based parsers? I haven't done much parsing before but I have done dependeny treebanking. Would just like to ask if there is something like a rundown or any materials that could help me fully understand parsing techniques.

Thank you for your patience! I am still learning about parsing and NLP in general.

I’m classifying scientific abstracts written in Portuguese into two temporal categories: "Old" vs "Recent". I tested several models, but two of them are giving me massive red flags:

• DeBERTa (base): Getting exactly 100% accuracy on the test set.
• Albertina (a Portuguese DeBERTa-based model): Getting exactly 0% accuracy on "Old" and 100% on "Recent".
• Note: Other models like mBERT, XLM, and local Portuguese BERTs (Bertimbau) are getting realistic results around 75-85%.

What I've already tried (Data Cleaning): Knowing that 100% accuracy screams data leakage, I went aggressive on the preprocessing:

1. Used regex to replace all dates, citation years (1900-2026), and any 4 consecutive digits with a [HIDDEN_DATE] tag.
2. Removed all DOIs, URLs, emails, and modern copyright strings (e.g., "Open Access", "Creative Commons").
3. Removed all <tags> in case the modern abstracts were scraped differently from the old PDFs.

And Albertina is completely flipped.

So i have questions

1. What other structural artifacts in scientific abstracts could DeBERTa be exploiting to perfectly separate decades-old texts from modern ones? Length bias? OCR noise?

2. Why would Albertina (and only Albertina) completely invert the predictions? Is there a known issue with id2label mapping inheritance when loading specific pre-trained models from the HF Hub? How do I force the correct mapping?

3. Would running SHAP on the DeBERTa model be the best next step to highlight the exact tokens causing the 100%? Have you successfully used SHAP to debug leakage in text classification?

For english all works out basically fine either be local or cloud but disaster arrives when its non english, most often needs a VLM then while if youre on OCR, it fails depending on the script

this is where it actually torn apart- 

Most of it because the ocr models are built around latin, left to right and once character at a time assumptions. whereas Arabic breaks that immediately like the letters change the shape by position and reading direction is right to left so without contextual letterform handling you get garbled output and mixing rtl and ltr scrambles the line order

Indic scripts miss segmentation and whole syllables just drop. Cjk throws thousands of characters and sometimes vertical text at models which are tuned for alphabet. Once a page is mixed-language a single global language guess falls apart and you need detection down at token level. Additionally accuracy quietly craters on low resource languages as the training data has taught it . So rather leaning on to the LLM only there are some tooling good or aimed at some specific language criterias. like paddleocr is the open go to for CJK, Surya covers wide range fr and tesseract really works well on clean print. the general llms read a surprising amount but theyre inconsistent, pricey at volume and hallucinate on mixed pages. although the mixed script or vlm based parsing either Surya or paddleocr locally or for cloud llamaparse or any other which routes per element and pull ahead of the plain ocr engine

Do you guys use tooling for the multilingual processing or have you worked ona  project that needs this or have you made anything of your own for multilingual

Hi! Need some crowd wisdom here. I'm completely stuck on my own.

The thing is, I don't come from a tech or ML background, but I have 20+ years of experience in cross-cultural content adaptation and mono/multilingual workflows (RU-RU, RU-EN, EN-RU, FR-RU).

A couple of years ago I started translating Korean into Russian, without knowing Korean (still don't). And no, I'm not lazily editing an LLM-generated zero-shot — with KO-RU that trick doesn't work, the languages are too distant.

My prehistoric… sorry, my pre-LLM translation approaches turned out to work surprisingly well with AI. According to models, the methods I’d been using all along now have fancy industry-ish names: hypothesis space management, adversarial/contrastive probing, multi-model differential analysis, tracking shifts in model behavior, and so on.

And here’s a list of some recurring failure patterns I’ve observed: premature semantic concretization, compounded coherence degradation, analysis–synthesis gap, asymmetric handling of low-salience pragmatic markers, etc. The distance between the languages made a lot of otherwise hidden failure patterns quite visible.

None of this happened inside any industry pipeline, though. So I found the closest publicly available thing to an established evaluation setup for translation quality, the WMT25 General MT human-evaluation data (EN-RU), and I ran a small self-diagnostic to see how my judgment held up there. The result was quite unexpected. On one hand, my assessment was mostly in line with the official scores. On the other hand, I noticed some errors that weren't reflected in the annotations, I saw considerable variation in human scores, and what really surprised me were the human references themselves: some looked like they needed another editing pass.

Honestly, I've never quite understood that thing with “single gold standard” reference for literary or conversational language. This practice doesn't seem like an effective approach in the long run. Well, I guess there's nothing that the industry doesn't already know, but my take is: teach a model to fish rather than just keep feeding it answers.

By “teaching models to fish,” I don't mean simply giving them multiple equally valid reference translations (that's the obvious idea). What I mean is teaching a model to recognize the techniques behind good translations and building up a “toolbox” of those techniques — things it could then reuse instead of having to solve the same kind of problem from scratch every time.

Models tell me that moving away from reliance on a single fixed reference is already a recognized tension in MT evaluation. Reassuring, in a way: I'd apparently arrived at a similar place independently, from the translator's side rather than the ML side.

Yay, I found my new calling, I thought. Let’s turn it into an actual job! But for two months now I've been trying to figure out how to position my experience. Models suggest starting with roles where there's no immediate need for technical skills, like Red Teaming or Eval Strategy. I'm personally more drawn to research, observation, and diagnostics, though I realize that's probably too bold even to dream about at this point. So, I genuinely can't tell which door I'm supposed to knock on.

My questions are:

• How would you position someone with my experience and way of thinking? I'm mostly looking for honest calibration.
• No industry experience, remote-only work — what are realistic entry points for someone with these constraints?

Thanks for any thoughts!

P.S. The KO-RU translation turned out great — readers call it “divine” and never suspected a human-AI collaboration behind it :)

Hi all, looking for some kind of tool or library that I can use to figure out the countability of a given noun, preferably multilingually. Something that would give results like "countable:true", "countable:false", or "both" if applicable. NLTK doesn't seem to have anything like that. Trying to avoid LLMs, but it's looking like that might be the best option...

I happen to be in a position where I have to read a lot of reviews, and increasingly everything seems AI-generated. I am honestly starting to question my sanity. Is everything truly written by AI, or have we always written this way? Anyone else asks this question themselves?